Does someone remember whether we had a question about quicksort here? Any implementation that is not 20x slower than Sort?

@LeonidShifrin Thanks for the reply. That's very helpful. In my research, the only language I have extensively used rather than Mathematica is Fortran, which doesn't seem to be popular in programming industry and I'm not a fan of it.

I have tried several times to learn and use python in my research. But I'm not familiar with the language and every time I tried to use, I found that the problem took me hours to code in python can probably be solved in minutes in Mathematica. And at the end I ended up switching back to Mathematica every time.

It seems that I have been using Mathematica as a "golden hammer", and I don't know how to pick up a new tool. Since you are a master of Mathematica, you can probably solve problems in Mathematica much faster. I'm wondering how do you learn a new language (for example python)? Do you have any suggestions? Thanks.

@xslittlegrass @LeonidShifrin In my experience, learning a new language or new paradigms only for the sake of learning itself, rarely leads to exceptional skills or expert knowledge. One key-ingredient is often that you just need to do something in the new language. No plan B, no cheating with Mathematica, no nothing. You will pick up something really fast by doing it because you have to.

Other than that, a natural drive to question your own approaches, to question whether you have the best or fastest implementation/method often brings you to the corners where you get exceptional skills.

@halirutan But Mathematica covers a lot of areas, and often I found the problems at hand can be solved in Mathematica. Could you give me your example how you learned a new language ?

@xslittlegrass Obviously I didn't started with Mathematica but with C, Haskell and Java in university. But to give you a specific example:

How should I have implemented the Mathematica Plugin without relearning Java? I had to deal with a situation where I had a working knowledge of Java but the next moment I had to deal with an API of several million lines of code.

@xslittlegrass @halirutan I actually learned Mathematica as a hobby first. Sure, I did use it for my research, since 1997, but only as an advanced calculator, so to speak. And, I didn't really program in anything for like 15 years, before I discovered functional programming and programming in general - through Mathematica and small toy hobby projects.

@xslittlegrass @halirutan So, the fact is, I had really no need for Mathematica in the capacity in which I learned it, at the time. It was just fun to play with it. But at the time, I was interested in AI and planned to use it for that, so @halirutan, you are right, there always is some goal.

@xslittlegrass @halirutan Then I learned C just because I decided that I need to know it. It worked well, I was a post-doc at the time and had some free time. I think I made all low-level mistakes possible, and ended up with my own libraries for memory management, dynamic arrays, high-level text processing and a bunch of other stuff. That helped me to understand C much better.

@xslittlegrass @halirutan As for other things - I learned either on the job, or from projects. I learned Java pretty much on the job, was a J2EE dev in a Java shop writing banking software, for a year. Thousands of classes, large projects, real OOP, lots of tools, from IDEs to SQL explorers etc. Can't imagine one can really learn Java on a small project. Python I learned recently, on a side web dev project.

@LeonidShifrin @halirutan I was first using Fortran in my second year of graduate study when I first got into research, and soon switched to Mathematica and sticks to it since then. Mathematica has almost become the only program language I use, with occasional use of Fortran through LibraryLink. Until recently, I went to a job expo and found that nobody is interested in Mathematica, and I think maybe I need to change.

@xslittlegrass @halirutan Same for js - I had a side project to write a kind of compiler from a trading strategy language to js / html, which ended up being 8000 lines of js, full of parsers, recursion, and all that. Was pretty fast - it was basically parsing a trading strategy code on every key stroke, and reconstructing the UI, and also back, from UI to the code. It was a UI wizard to help traders write trading strategy code.

@xslittlegrass @halirutan In any case, one thing is for sure: one needs some real project to work on, maximal freedom to make decisions (right or wrong), and enough time to learn from one's own mistakes and analyze the code - one's own and the code of others. Which sadly means that it is rather unlikely to learn effectively on the job - unless you are already very good and are just upgrading.

@xslittlegrass @LeonidShifrin Yes, I understand you but I don't have further advise beside that what is written between @LeonidShifrin's lines: do a project.

@halirutan Yes, I agree with you. The problem with this approach is that, if you aren't really good already, or don't have really lots of time, you may pay the price of taking some things for grunted, and that would limit the depth of the understanding of the language. I speak from experience here, since I had a luxury to cut Mathematica to the bones when learning it, but other languages I had to really learn fast and use right away, and I do feel those gaps and how they limit me, all the time.

@LeonidShifrin Exactly.

@halirutan Also agree. The problem is, one really needs time (lots of it), to get those exceptional skills. For example, I really like Python, js and C, but just don't have the time to have a real upgrade in any of them - even when I have to massively use them.

@LeonidShifrin ... and in today's jobs in the free market, you will almost never get this time. Get things done is more important.

@halirutan In my experience, when I need to use the language and am short on time, I don't really advance much - I just go to SO, pick the answers, get the job done, move to the next thing. This type of work does not advance your skills all that much. Much better is to play and tinker with things, and their relations. Have time to discover things yourself, to break things and tear them apart.

@LeonidShifrin @halirutan OK, I get the point. So I guess maybe I should start learning the new language from a project that is not related to my research, so that I can have the time to think and make mistakes. For all the tried I had, I attempted to use python in the way that completely replaces of Mathematica in my research, to force me to learn. I now see why that's a bad idea.

@halirutan True. You can't get free time, you can only buy it.

@xslittlegrass Probably one tip: When we write algorithms for project partners, it's almost never the case that they have Mathematica. So what I sometimes have to do is to re-implement things I already have prototyped in Mathematica in another language. Why don't you take a rather small (numerical) method and re-implement it. You say no one has Mathematica? Fine, make your work available in a more common language.

I once had a complete quantification of finger joint laser images ready in Mathematica, but for a research project this algorithm had to work standalone. So I re-implemented everything, starting from the image and signal processing, camera exposure control and UI in C++.

@halirutan I've taken that route sometimes; I write some numerical algorithm in idiomatic Mathematica for proof-of-concept, run tests, and if it works, take on the task of translating it procedurally to the languages it's actually needed in.

This is how you learn how much work it takes to re-implement the stuff you take for granted every day in Mathematica :-)

@xslittlegrass I agree with @halirutan - porting from the language you know is a good and well-defined problem, that can allow you to learn quickly.

@halirutan ...and you gain even more respect for how Mathematica was designed. :)

@halirutan Wow, that sounds impressive! I wouldn't want to touch C++, but perhaps wouldn't mind knowing it, if that knowledge could be acquired during my sleep in one night, without pain and without my soul being lost or sold to the devil :)

@halirutan So, what's your feeling about C++? For all I know, it is one of the most horribly designed languages.

@LeonidShifrin Qt is C++, Intel's Threading building blocks are C++... I really did not had a choice there :-)

@halirutan What about plain C?

@LeonidShifrin I can honestly say, I don't know. Understanding templates alone (and yes, I mean all the details of their specifications) would take ages. So I would probably need 8 more years of those 10 years you need to learn a language completely.

@LeonidShifrin I guess I feel the same as you about C. The restrictions compared to C++ make it more pleasant. One thing about C++ I can't stand is that you still can do everything you can in C. What a horrible mixture.

@halirutan The funny thing is that just today I was in a discussion with someone about C++, and then (re)read that thread with Linus Torvalds bashing it. In fact, if you read other responses, together they give quite compelling arguments, although tied to git development.

Additionally, I cannot say that I really like the code of e.g. the boost library.

@halirutan I just have a gut feeling that C++ produces a lot of accidental complexity, and also has too many gotchas, so in practice one would have to spend many years learning it, just to be sure one is using it right - which seems a bit too much effort.

@LeonidShifrin Oh my god.. this is so awesome.

@halirutan Oh yeah

Hi @R.M.!

I spent a whole morning the other day reading those threads

@LeonidShifrin I like people with definite opinions. Even better, when I share them.

@halirutan Exactly the same here :)

And everyone that had to debug a template error will instantly know what he is talking about..

@LeonidShifrin Hi! Long time! :)

@halirutan Yes, that's exactly what my gut feeling also is, for what little exposure I've had to C++. Those guys in fact desribe the C philosophy pretty well in those threads, and I like that and agree with that.

@R.M. Long time indeed :)

@LeonidShifrin @halirutan Thanks a lot for the advice. They are very enlightening! Just curious, how many hours you guys have for sleep on average every day? Do you have time to workout everyday?

@xslittlegrass -4 for halirutan, from the looks of it

@R.M. @xslittlegrass and -6 for me :)

hehe :)

btw, your book was featured on Hacker News a week or so ago. Did you see it @LeonidShifrin?

by featured, I mean I saw it on my front page for a while

@R.M. No, I didn't. Do you have a link?

@xslittlegrass Well, my clock tells me that it is 4:56 here, and 5:56 on Leonids side of the planet. Last night was tough as you might have seen in the other chat.

@LeonidShifrin news.ycombinator.com/item?id=10280100

@xslittlegrass And yes, I have to have time for workout several days a week.

Otherwise I go crazy.

@LeonidShifrin Not much actionable info there in the comments other than the usual expected stuff, but 23 points and front page is a good showing! :)

@R.M. Interesting... Thanks for the link! Although, as expected by now, the discussion was mostly about other things, in comments.

@R.M. Oh yes, forgot about the points. Indeed. That's nice! Actually, when I look at the book now, I feel it is really rather naive in many places, and also lots of important stuff not covered. I can do much better now, and I plan to return this debt to myself, sooner or later :)

@R.M. So, do you still use Mathematica these days at work?

@xslittlegrass I still miss Mathematica as a "golden hammer". I doubt I can be as productive in anything else. However, I've been writing a lot of Python and Scala the past year and a half with very little Mathematica and to be honest, I wouldn't have gotten far had I had access to Mathematica at work.

@R.M. Ok, you answered :)

@LeonidShifrin Haha, see above comment :P No, none at work. I still use it via my old license at home :)

@halirutan Well, I have nothing to complain about, I get 6+ hours of sleep everyday.

@R.M. I see. In fact, I've been also using Python quite a bit lately, for some side projects in the spare time (when I had some). But that was mostly Django, not rocket science. How do you like Scala, and why do you need it? Is this a Java replacement for something like big data handling? I've heard that Scala is frequently used in such role.

Wow, we have full house now! Hi @Szabolcs!

@LeonidShifrin This cannot be the real @Szabolcs. As far as I know he is still in Europe and 5 in the morning is not his time :-)))

@halirutan It might be Mr.Wizard impersonating @Szabolcs :)

@LeonidShifrin Yeah, I wrote a recommendation engine for use in production (exposed as a set of APIs) and this uses some standard ML/big data tools like Spark, which is written in scala. This is a complete end-to-end user-facing product and quite frankly, would've been near impossible in Mathematica (and prohibitively expensive/slow even if it were).

@R.M. Wow, that sounds impressive! So, did you also write some SML? This sounds like a fantastic project. I've been interested in custom search and recommendation engines for some time, but the lack of time didn't allow me so far to go beyond toys.

I like scala. It allows you to think functionally, move from idea to code fast like in mma, create DSLs easily, etc. and at the same time have access to java's oo and libraries. You can pretty much drop a jar into the path and import it in scala.

@R.M. Did you pick it as a part of your stack or was it used already at work?

@LeonidShifrin Haha, no not that ML. I meant machine learning. :)

@R.M. Oh I see. So, did you have to learn that field too? This is another one of the things that I wanted to pick up for a long time.

@R.M. The funny thing is that if you didn't use the word "standard", I would've interpreted correctly, but "standard ML" triggered a different interpretation :)

@LeonidShifrin It wasn't/isn't used at work. The engineers use python for everything else. This is the only piece in our stack that runs on scala/jvm and I chose it because it was stable, scalable and had good library support. Even though I knew zero scala and was good in python at the time the decision was made, it was the right one.

I picked up scala very fast because of prior exposure to java and FP via mma. I think you'll enjoy it because of your strong background in OO + extensive experience in Java/Mma

@R.M. Well, that sounds just great, my sincere congrats! It sounds like you've made an optimal path: picked the right technology, probably had enough freedom to learn and use it properly, made a real product, and surely learned a ton of things when doing it! I must say that I envy a bit, in a good way :) My last year was not nearly as productive.

@LeonidShifrin I'm wondering what's your daily job look like in Wolfram Research?

You are still there, right?

@R.M. Scala has been on my wish list for at least 5 years. The problem is, you need to have a real project and time to work on it.

@xslittlegrass Yes, I am still there, but I am on a long break due to a number of long-standing health issues I need to fix. So right now, I am just trying to deal with those.

@R.M. I've heard that his course is quite good. The author of technology is not always the best person to describe it, in general.

@R.M. This was also smart because now you don't have to learn Java, ever.

@LeonidShifrin Hehe, yes, that was partly the goal ;)

One of the complaints people have when they join a scala shop (e.g. Twitter) is that because scala supports multiple styles of programming, it is chaos when you try to get an "expert in scala" from a Haskell/FP background to work with an "expert in scala" from a java/verbose OO background. Neither can understand the other person's code :P I haven't faced this yet since I'm the only one that uses it, but something I fear I might have to face eventually

@R.M. Well, then you have to become an expert in Scala from both sides, by then :) But actually, it is rather hard to get a full appreciation of OO unless your project is large enough (a few hundred classes at least). And OO is certainly not always the best tool to structure code.

@R.M. One of the things I appreciated in Java was its interfaces. That style of separation of interafce and implementation they allowed, and programming by contract. That lead to some really clean and reusable code. Mathematica would benefit from some such construct quite a bit, but of course that would require some version of types. I do use such constructs there, but they are not in the language.

@LeonidShifrin Could you elaborate more on the separation of interface and implementation. Isn't that the functional program in Mathematica is trying to do that the separation?

@LeonidShifrin Types and static typing are something I really started missing after working with Scala. I might be realizing this much later than others here because I've had zero experience w/ languages like C, C++, C# and only little experience Java and Haskell — certainly not w/ large projects. Types and static typing make refactoring such a painless task when the code base gets larger!

@R.M. Returning back to Mathematica, I wonder, what would be the main obstacle for implementing your system in Mma? I mean, apart from the price. I could think of several things, such as speed, stability, deployment issues, may be not enough tools in the language to help structure lerger code bases - but what would be the most important one in your case?

@R.M. Funny, we again had similar things in mind :)

@R.M. I've been lurking but I just had to +1 this one.

@xslittlegrass No, Mathematica's FP doesn't really enforce it. You need to impose some constraints, which you probably can only cleanly do with types.

Hi @WReach!

@LeonidShifrin Hello!

@WReach It's been a long time :) And you are exactly the right person to join now!

@LeonidShifrin OK, but at the function definition, we can have patterns to restrict the argument type, like f[x_Integers]. Is this a simulation of the type?

@R.M. Actually, one thing I realized some time ago is that Mathematica makes it not that difficult to add types. I use some versions of types (like weak types induced by patterns, or even strong types) all the time in my M code, and I think it is the only reason why I have relatively few bugs and some sensible way to maintain reasonably sized code base.

@xslittlegrass Funny, it looks like we have some collective mind during this chat. I've been just finishing the previous comment which was exactly about that. Yes, we can, of course.

@xslittlegrass And it is a simulation of type, for sure. Perhaps, the biggest problem is that most of the time it doesn't make the code run faster in Mathematica. But tyoes are also useful for correctness, not just for optimization. Perhaps, the former is even more important.

@xslittlegrass What I meant when I mentioned interfaces was that they are a language feature, enforcing that separation on the level of the compiler, and at the same time allowing polymorphism (any class that implements an interface can be used where an interface is required). This is useful because it explicitly forces you to abstract away things which are implementation details.

@LeonidShifrin 1) Deployment. I built the whole thing and deployed it as a web-service on our cloud infrastructure so that the main production app can call it as an API. This was relatively easy to do and integrate, with my stack (or in python, had I chosen it) but impossible to do this with M without some prohibitive licensing or going through W|A or some Wolfram* product. (with the latter, there's the unfamiliarity involved as well)

2) Stability. During the time that I was using M every day for 100% of my work, there was little to no visibility into when a kernel/FE might decide to die. No logs, nothing. And the crashes are not predictable either. Not suitable for a product with strict uptime/reliability guarantees. 3) Unpredictable unpacking. 4) Too much Entityification and W|A calls which slows down the processing and increases network overhead. I cannot simply strip out core M and use it.

@LeonidShifrin I see.

@R.M. This is pretty much exactly what I was guessing. I mean, I was thinking of exactly the same set of possible issues, just wasn't sure about the order. Do you think you paid with some extra time for the parts where you could develop M code (much) faster that say Scala code? I guess, even if so, that time was probably saved in other places, right? And which language did you end up using for prototyping - Scala or Python?

@LeonidShifrin You must have access to the source code of Mathematica, right? I'm wondering how large is the source code?

@xslittlegrass You can imagine :) It has been developed for more than 25 years by many people. Actually, in Mathematica docs they give the numbers - millions of lines of code, both C and Mathematica.

@LeonidShifrin hard to say, but my answer would probably be no, at least for this specific case. The use cases where I miss M the most would be ones where 1) the issue/project involves lots of exploratory coding without a specific structure in mind, 2) need to evaluate different strategies really fast 3) solution/approach is very cross-disciplinary (e.g. image processing, graph theory, statistics, visualization, db access) and I can move fast without having to learn/evaluate a new library and its api

@R.M. Yes, this also very much aligns with what I would think to be best uses of M. So, I take it as in this project there weren't too many cases like that. It sounds like you must have known pretty well at least the direction and what you were planning to do there.

@LeonidShifrin Yes, I did some preliminary whiteboard planning to decide the general direction and what needs to be done and once that was set, the remaining decisions were fairly easy to make

@R.M. Sounds really cool. Congrats, once again :)

Thanks :)

@LeonidShifrin I'm also considering working in Wolfram Research, if I can get the chance. I can imagine the excitement to develop such a wonderful tool with a group of intelligent people. You must have worked in other companies, what makes you stay at Wolfram?

@xslittlegrass Mathematica is a technology I know best, I like it, and the projects I work on are interesting. But still, it's not as simple as that. I think, this is always a very personal choice, and what works for one person, may not work for some other. I do know some people for whom it worked even much better than for me. Working at WRI is surely a great experience, you just have to make sure you pick something that's right specifically for you.

@LeonidShifrin I'll throw in a couple of thoughts concerning the suitability of languages like Mma for larger projects.

Tool support for reasoning about a large code base is important to me. Java's simple-mindedness has one advantage -- such tools are feasible. For example, one can highlight a method in the IDE and hit a hotkey to see every possible call stack involving that method.

In Java, virtually any use of reflection defeats these tools. And Java reflection is not even a kid's toy when compared to Mathematica's symbolics. "Dynamic" languages like Mathematica give us fantastic notational convenience, but at the cost of losing such tooling. When programming in-the-large, I think the cost is too high.

@WReach Thanks for sharing your thoughts. In fact, from my own modest experience, I have similar feelings. I did have the chance to appreciate the tooling you mention, for Java, working in IntelliJ (which is fantastic), and then Eclipse (which is just fine). I even worked for a few months with JDeveloper (we used Oracle stack on one project). In all cases, the tooling saved countless hours of work. Something I do miss in Mathematica.

@WReach What I was really surprised with, though, is that at least some IDEs (IntelliJ) approached close if not similar degree of tooling support for dynamic languages (Python, js). The way PyCharm handles Python projects is just amazing, I didn't even think they could go that far. Of course, dynamic languages have inherent limitations for code analysis of this sort.

@LeonidShifrin IntelliJ is certainly an amazing piece of work. I particularly admire how it will support such tooling within code snippets embedded within strings in a different language! Tooling always improves, and even Java's support would have seemed like science fiction not too long ago.

Some language features have the potential to tangle up code. Closures run the risk of being the contemporary version of "Goto considered harmful...". And Mathematica can create some interesting mazes after the parser has rewritten the expression, the symbols have been rewritten, up-values trigger and the built-in rules apply some unexpected optimization :)

@WReach Yes, very true! IntelliJ is indeed amazing (their main dev. office is in my city, b.t.w.). They started from Java, but then proved that what matters is to understand what developers need, and that matters more than a specific choice of language. And the state of tooling indeed improved dramatically, and keeps doing so. One thing I noticed is that no matter how powerful the computers become, tools always catch up and use up all that power. Well, at least not in vain :)

@WReach Also true. In Mathematica, however, closures are not that popular. Perhaps, this is a cultural thing, plus Mathematica doesn't have a "sanctioned" way to create closures over mutable state, since Module - generated variables still feel like a hack.

@WReach But returning to large-scale code bases, and the complications of dynamic symbolic languages and Mma in particular, do you still find ways to use Mma in your projects / workflow?

@LeonidShifrin Mma remains the first tool I grab when faced with an improvised programming task -- even after losing a few toes to the bug minefields of recent releases ;D I find that as long as a human remains in the loop (i.e. me, the user reviewing the results obtained in real time), then Mma is very productive. But once the code base grows to a size where most of its operation is without such supervision, I start feeling the pull to move the task to a strongly-typed language.

If I try to characterize the reason why, it is because Mathematica's programming model is opaque in various dimensions: performance model, increasing use of heuristic behavoiur instead of well-defined rules, and the "dynamic" features we were discussing above.

@WReach Yes, sounds logical. So, once again, this confirms that Mma is most useful for interactive work - which is logical because it always has been its very strong side. My own recent experience with developing the (not yet quite finished alas) Streaming module (which is by now about 15K loc, if I inlcude the testing module) led me to similar conclusions. Actually, I made most of my Mma code there de facto strongly typed, and that was the only reason I could keep it manageable.

@LeonidShifrin One of my pet mottos for programming in general is to "get the defaults right". I want my programming language to have pattern-matching, unification, reflection, ultra-late binding, macros, programmable compilation, etc. etc. etc. -- but not by default. 90%+ of the functions I write in any language are plain old Java-esque methods that take some simple parameters and produce some simple result. This is amenable to great tooling and should be the default.

But not every function needs to public, virtual, dynamic, and late-bound. The language should not make such an assumption without an explicit statement to that effect. </rant>

@WReach I haven't seen the opening tag :)

@WReach I don't have such experience as you, but I fully agree with this.

@LeonidShifrin look back to SO in oh, about 2009 :)

@WReach Oh, I see :) So, it should require a lot of RAM and CPU power to parse the body between the opening and closing tags, then :)

@WReach Besides, then, didn't you close the tag prematurely :) ?

@LeonidShifrin especially with regex. I'd better re-open the tag in case I forget later... <rant>

@WReach It looks to me that today is a special chat event. It has been several times that people were reading other's minds and posting colliding comments, during this chat session. Something is happening :)

@WReach And yes, the regex :)

@LeonidShifrin Well, it is nowhere near as late/early here as it is over there, but I've got to pack it in for tonight. Thanks for the chat.

@WReach All right, it was great to chat! It's 8 a.m. at my place, and I'll go get some sleep

@WReach Funny - this happened again

11 pm here. Good morning!

@WReach Yep. Good night! :)

@halirutan @Leonid I suspect "Power Nap". Apple laptops will wake up in the middle up the night to update/backup/download mail, etc. I was definitely sleeping.

I used to use Mathematica for Windows, and it there was a built-in tiling window manager that I found useful every once in a while. Is this still in the Windows version? It isn't there for Linux, so I was wondering if anyone here uses an external tiling window manager, and if so - which one works well with Mathematica? (I imagine this would be marked as off-topic if I asked on the main SE)

@Karsten7. DSolve has a huge improvement on PDE solving in 10.3! Really exciting!

@LeonidShifrin "it not that difficult to add types" ... I am not sure i understand, please when you have time could you show some basic code examples involving "types". Thanks.

@WReach @LeonidShifrin I went to bed a bit earlier but followed read the chat-log now. @LeonidShifrin the number of the tools we have for java development are exactly the reason, why I like Java so much. Although the language is very restricted by this compared to e.g. C++ templating or Mathematica, it creates the possibility to provide stunning code-refactoring, auto-importing, iteration support, and most important the possibility for the IDE to tell you on spot whether your code will compile.

As you already said, it's shocking how many code-insight features IntelliJ already has for python and other languages, but for Java it's just frightening to see this in action. This is btw, the difference of the 3rd semester halirutan that tries to write a java project assignment in a plain editor and me now. Back then, I just thought whether they are out of their freaking mind to expect me to know all the import libraries or look up even the most simple tools on Lists.

Now, I don't have to care about it any longer. The IDE is doing most of the stuff. The import section is always folded and I never look at it. You have documentation and the whole java sdk at hand and you can look up parameters and implementations specific to your current line of code. Overwrite an abstract class and the IDE will know what methods I need to override.

And this really pays of for larger projects and makes up for that you have to write really really verbose code.

One additional point. The plugin has currently about 20k locs. @xslittlegrass pointed me to a performance issue when there is an error in the file. This issue could have been caused by a million things: anywhere in the parser, anywhere in the API, anywhere else. It literally took me less than 5 minutes to pin the exact method down by using a profiler. That was one of those moments where I was really happy I did not have 20k lines of Mathematica.

yes i love java for the same reasons, the tools are fantastic. i always get slightly upset by people hating java around me, when it really starts to pay off once you hit the bigger codebase size at like 200k LOC (for lack of a better metric)

People, I am looking for one last re-open vote for this question. I want to get it re-opened so I can write a community wiki answer for it. See the last few comments to the question to see why.

@m_goldberg No problem. Done

@halirutan. Thanks a lot.

@m_goldberg So, are we not doing question text tagging for confirmed bugs and fixes anymore? Personally, I like that format better than community wiki answers stating that same info.

@dionys We can still do that, but the CW answer does ease the number of unanswered question a bit.

@halirutan I agree with most of what you said, but not fully agree that IDE totally compensates for verbosity. The problem is that, in a verbose language, it is hard to make really good, simple and powerful architectures, and it is hard to use the powerful abstractions. So, yes, IDEs help to an extent, but I have no doubt that languages supporting more powerful abstractions (like e.g. OCaml, Haskell, or Scala) would yield better designs and simpler code.

@halirutan I didn't realize that the plugin's code base is that large. That's impressive. And your story with the profiler and instant location of the bottleneck is very impressive, as well. I certainly do agree that the combination of Java + tooling offers very significant advantages in a number of cases. I think that as always, it boils down to always use a right tool for the job. B.t.w., it would be interesting to see how much less code it would've been if you wrote the plugin in Scala.

@J.M.isback. The unanswered statistic is a valid concern, but ideally all answers (cw or otherwise) should add something extra that is pertinent to the issue in the question. If there's nothing to say, I'd rather they stay "unanswered". (Just my $.02).

@dionys That's fair; an answer should have some value added. Perhaps a screenshot of the fixed version now behaving as desired?

Hey guys, is VarianceEquivalenceTest[{data1,data2}] the same as VarianceTest[{data1,data2}]?

@LeonidShifrin not everybody learns by studying the structure of things. Some learn better by tinkering. Obviously, you are the first type, but others aren't.

@halirutan hah, I started thinking "Hm, Torvalds seems like a jerk" and then read that... He's not wrong...

@acl He can be at times, to be fair. But, it is often offset by his being right most of the time. ;)

@J.M.isback. In general I don't agree that being right means you can be a jerk. However, I think anybody who's touched C++ knows exactly what he's talking about and that means he can be a jerk as much as he wants.

@acl "in general" - certainly. Still, some of his old newsgroup postings have nevertheless proved... popcorn-worthy.

@J.M.isback. some of the Mathgroup threads also were interesting, shall we say.

Some very strong and very unsupported opinions colliding.

@acl If that was really a reply to me, then you are definitely wrong about me. I learn by tinkering and breaking things apart, not from studying the structure of things, in the first place. In fact, in my previous posts, particularly here, I made this very clear.

Hm. I thought they had fixed all of the *Data links in the documentation. ComputationalComplexityClassData is broken though. reference.wolfram.com/language/guide/…

Anyone can tell me what is the equivalent of a Welch t-test in Mathematica? I have searched everywhere and can't find it :\

@halirutan Yep, that's why I never had the patience for the boost graph library ... it has a lot of "flexibility" (read: complexity) in that you can even use any representation you like, adjacency matrix, edge list, etc. whatever you make a class for that conforms to the right concept. You decide to use it, after all it should be flexible enough for anything. Then you discover that it takes days to learn (and avoid cryptic template errors) if you want to do a simple but very specific task.

Then if you manage to do it, you finally realize that the system is not transparent enough to understand well why something does or doesn't perform well.

It would have been quicker and more effective to start with a simple, problem specific data structure to accomplish this simple and specific task ...

Bu there are easy to use C++ libraries too. Qt was nice last time I used it (years ago).

@Szabolcs And even Linus Torvalds acknowledges Qt by using it in his only other project on GitHub beside the Linux kernel.

@LeonidShifrin A proper estimation is 12.5k loc

-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Java                           362           2817           9087          11644
XML                              4             99             71            483
Objective C                      3             54              0            298
HTML                             6              7              0             52

I had only counted lines before (and java only) which gave something about 24k. I guess I didn't realized that I'm that verbose in my comments :-)

@LeonidShifrin I understand your position. The problem is that two things here are negatively correlated: powerfulness of the language <-> code insight features of the IDE.

But I have to admit, that a big big part is whether the language is strictly typed and whether it is possible to tell the type of an expression by knowing rules of the language (opposed to the need to really evaluate it to know the result type).

@halirutan MUMPS?

@Szabolcs That's the exploit that comes with the plugin, secretly stealing your data.