Was curious about my own files

listFunctions[file_] :=
 Cases[Import[file], Cell[data_, "Input", ___] :> data, \[Infinity]] //
    Map[Cases[
      ToExpression[#, StandardForm,
       Hold], _Symbol?(UpperCaseQ@
          StringTake[ToString@#, 1] &), \[Infinity],
      Heads -> True] &] // Catenate

listFunctions /@
     FileNames["*.nb",
      "c:\\users\\michael\\google drive\\notebooks", \[Infinity]] //
    Catenate // Counts // Sort // Reverse

Pretty rough way maybe, but first I came up with

<|List -> 46200, Hold -> 7889, TypeSystem`Atom -> 6908, Slot -> 6802,
 Rule -> 4368, TypeSystem`AnyType -> 4104, Function -> 3765,
 Part -> 2706, Set -> 2680, String -> 2319, Integer -> 2224,
 Null -> 2006, TypeSystem`Enumeration -> 1820, Blank -> 1683,
 CompoundExpression -> 1591, Map -> 1497, Times -> 1169,
 Pattern -> 1028, All -> 790, Plus -> 744, Real -> 701,
 TypeSystem`Tuple -> 674, If -> 652, Select -> 617, Power -> 600,
 Import -> 505, Length -> 490, TypeSystem`Vector -> 484, Span -> 458,

List, Slot, Rule, Function, Part, and Set do feel like my most often used

4 more points and I'll be a newb, not a freshly hatched chick looking for a dog that looks like a cat with a fire hydrant

Has anyone seen or know of a way to create a graph similar to this one in Mathematica digitalstudies.org/ojs/index.php/digital_studies/article/…

@MichaelHale Is this related to Szalbocs' comment? Because I don't think "used most often" works as a subset of the language.

@EmilioPisanty Did my comments help? Are you exporting the result to EPS/PDF? If yes, there's always the option to outline all glyphs in the result, which will make them render the same way. Whether this is a good idea depends on the specific figure. Font outlining can be done with Ghostscript (that's what MaTeX uses Ghostscript for as well).

Does Mathematica have built-in arrowhead shapes? Or do I have to create my own?

@Szabolcs aren't there some in grpahDataWhatever? or chartData?

@Szabolcs e.g. GraphElementData[{"HalfFilledArrow", "ArrowSize" -> .1}]

@MichaelHale do you not feel a self-answered question on listing frequent functions across many notebooks would be useful? I certainly do.

#dailyFun

@Szabolcs I'm exporting to pdf. Sorry I didn't reply, I'm in the process of submitting the thesis.

@EmilioPisanty Good luck then and don't fuss too much with formatting!

I think I'll just go over all the figures after I've submitted and MaTeX everything just for consistency

but it's still not great that you need to do that to get it looking right.

Take e.g. this figure

dl.dropboxusercontent.com/u/60184856/figure5Kg.pdf

looks mostly OK on evince

but on a laser printer you get very different weights on the ticks w.r.t. the labels

@MartinJohnHadley Ha it could be a fun excuse to generate some of Wolfram's favorite word clouds to see individual coding styles. I'll post a q&a in a bit, just woke up. Seems like it was digging into datasets though. Not sure why since I was supposedly grabbing input cells. I'll look at that and options to restrict function list before posting.

@Szabolcs oh, and just for full info, that's font size 9 in both the ticks and the MaTeX call.

I honestly cannot see the weight difference in your screenshot, so I cannot tell what is MaTeX and what isn't ... But I know that there can be a difference, even if hidden by rendering at a given DPI.

I thought you were referring to "soft recollisions", which is in smaller font

@Szabolcs I could not see the difference either, but my vision has gotten a little fuzzy with age.

Should this Print or not? ExportString[Dynamic[Print[1]], "NB"] vote now

@MichaelHale it did look like Dataset internals were showing through, but I didn't put it past you having them deliberately used for some nefarious reason. Wordclouds are definitely a must, please do ensure that the wordcloud is shaped to the most popular function name in your directory. Cheers and have a good day :)

@C.E. I was assuming @Szabolcs and @LibertyTrooper were trying to streamline learning the language. Then I think learning the functions in order of most used (sprinkled with others as you need them) is a reasonable base ordering.

@MartinJohnHadley Final results. Post on the way

@MichaelHale amazing. Without the word cloud I couldn't have understood this information +1

@MichaelHale @MichaelHale You are correct. Yet beyond just learning the language, its about learning how to use that basic subset efficiently too. For example, for handling errors, logging, etc.. Without such it easy to mistake it as a batch language oriented towards Mathematicians working on their publications.

I know this information is scattered around this site. However, it can be daunting to locate it because everyone's vocabulary differs.

Perhaps I'm just making noise. And i certainly cannot speak for @Szabolcs . I probably will realize that what I'm looking for is something that I've already seen somewhere and just did not properly internalize. However, if this is incorrect, then maybe there is a way to assemble that solution with input and collaboration of all whom are here.

@LibertyTrooper But isn't it precisely because everyone's vocabulary differs that you won't find what you want in a single source? The function vocabulary necessary to complete the projects you want to do is probably a little different from anyone else's.

@MichaelHale I realize I'm probably just making noise. However, If you look in the Fast Introduction for Programmers, I don't believe you will find a chapter devoted to the basics of error handling. Of course, perhaps I'm overlooking or overcomplicating it. And, perhaps I'm mixing concerns rather thank keeping them separate.

A discussion of a basic language subset including those things that address idioms that crosscut languages is something I think would be useful.

I know in the back of my mind that sooner or later, I'm going to look for try except catch finally. Once I get past understanding why things do not work.

Anyways, I am one of those people who, in discussion, usually find the answer staring me in the face resulting in many, many Face-Palms.

@LibertyTrooper I don't know if I've ever used the Throw/Catch functions in Wolfram. I've used Check a couple of times I think. But other than that I just wait for errors to happen and then if them away or replace or delete them in the results.

@MichaelHale So you look at every error and then deal with it specifically?

@LibertyTrooper A lot of times I don't even deal with them. Maybe the two aspects of Mathematica that have saved me the most time are the notebook interface and the fact that it can often keep processing a list of things even when some generate errors. Then I can decide if I need to deal with them or just ignore them. Every other language I've used like C++, C#, F#, Python, R, etc will just abort the whole program.

I use an approach that assumes sooner or later something 'bad' will happen. Then, the first time that something 'bad' happens, i won't have something that is potentially irrevocable occur.

Then you have to write very general error handling, like wrap the whole thing in a catch, which will let it finish often but doesn't help you much. Or you have to fix each error as they occur. But let's say it takes 6 hours to process, and it's only 4 hours in that it encounters a weirdly formatted input. Then your whole night of results might be wasted. Trying to guess all of the potential errors in advance gets tiring and a lot of the time it is unnecessary.

@MichaelHale I think thats valid if in that case you are writing that 90% of programs which are for a single purpose. And maybe this is where we have differing expectations as you mentioned. What I want is something that will be used by a lot of other people. Maybe I'll provide a way to just eat the exception or ignore the error after allowing the user to eyeball it. There still needs to be some way to say 'Hey, something I didn't expect happened. What should I do?'

@MichaelHale But this is a tangent to what the original comment was which concerns a reasonable subset of the language for general purpose use. Which I interpret to mean addressing those crosscutting idioms.

cross cutting concerns that are expressed as idiomatically as a result of the idiosyncrasies of language.

Well it generates messages, and a nice feature in v11 is that it saves the stack trace for each of them if it is a message you need to investigate. If you run your code in debug mode you can tell it to automatically break when a message is generated, similar to many mainstream IDEs. Or you can use Check to do something whenever a message occurs, which I do occasionally.

I was shortcutting a lot in my head.

@MichaelHale Thanks for making me think how to better express what it is that I think and mean.

Haven't yet had the requisite amount of coffee this morning. Heck, I might even go back to bed.

I'd like to hear @Szabolcs thoughts too.

@MichaelHale And, today I'm going to go through the Fast Introduction for Programmers again. Maybe I'm simply missing things.

I don't think you are missing anything. I just don't think you are going to find an answer staring you in the face like you say always happens with this one. To learn more you need to write more code. As far as what code to write, you can maybe find exercises, but if they are too different from your goals then they will be boring. So I recommend just trying to write the code you want. Maybe your projects are too ambitious that you don't know where to start though.

@MichaelHale To be clear, I always assume that there is an answer to my question and that I just haven't looked hard enough or I've misunderstood one I've already read. Moreover, I chimed in on @Szabolcs statement with general observations. Not because I've a specific application. Generally, I read the questions that are posted here and then try to come up with answers. I think thats an excellent way to learn. Thanks again for your hlep.

Anytime

Mathematica boggles the mind. Its capabilities, as demonstrated by many examples including the one you just posted, are otherworldly. Its hard not to want it to be a general purpose language when there is so much that it can actually do.

I consider it a very general purpose language, but the trade off for having such a nice language design and large function library are that it's not free to run your code on as many computers as you want. I certainly don't consider it a domain specific language. Do you? I'm relatively greedy though. I prefer writing code for myself. I only write code for others if they pay me or if I think I'll learn something from the quick exercise. So the licensing limitations don't bother me at this point.

Or do you just mean it's not a strongly typed language, which is considered useful for large applications? Wolfram always comes back with the Wolfram Alpha example though.

@MichaelHale streamline learning the language, exactly

some complaints news.ycombinator.com/item?id=10766462

The second one reminds me of how I feel using Python, not Wolfram. Like two weeks ago I needed to yield a list of results in Python. And I wanted to put the yield inside of a lambda inside of a map I was doing. But it turns out you can't do that because a yield expression is a special type of expression that can't go inside of a lambda expression. I'm used to Wolfram where that would just work because it's all symbolic expressions.

And I certainly couldn't reason my way to what finally worked in Python. I had to use Google searches and sifting through the documentation.

Gotta run some errands, back in a bit and I'll read through some more of them.

There is a very famous book about JavaScript, called "JavaScript: the good parts", in which the author introduces JavaScript entirely without mentioning some of the features of the language. He then wraps up the book by mentioning also the "bad parts", cautioning the reader against using them.

However I can't find a similar subset in Mathematica. People have sometimes been told not use procedural constructs, and that's in that category. It probably makes it easier to understand what coding in Mathematica should feel like, but it can also cause a lot of trouble for new users who will then set out to solve problems functionally in a very contrived way, when in fact procedural programming was the right choice for that situation.

Perhaps Mathematica should not be taught as a golden hammer: Perhaps just the functional parts of Mathematica should be taught, along with problems that it solves elegantly. That subset would be able to solve a wide range of problems, and do it well.

I think it really helps if new users are first introduced to the parts of the system that are farthest away from anything procedural. A good place to start imho. are replacement rules.

Aside of WL/Kernel talk - For me the biggest concern is that fundamentals of the system are obscure. There is no way for a newbie to learn reading-parsing-evaluation-formatting-typesetting loop without spending a lot of time on SE. And even here the knowledge is mainly based on experiments... Moreover, stylesheets' resolving scheme is not documented.

Docs only treat about the surface of GUI generation, surface not useful at all for bigger projects. Everything more complex is vague or not there. The more I use it the less I'm eager to recommend it to others. Not that I don't like it but I because I want to be fair.

It is really annoying, for a beginner, to play with loosely connected black-boxes that were advertised as intuitive and flawless.

There is tutorial/TheMainLoop and tutorial/WolframSystemSessions. I'm pretty sure I saw those before I saw Mathematica Stack Exchange.

@MichaelHale it is Kernel/evaluation centered (nothing about boxes there), full of gaps (autoload is abort protected), and inconsistencies (symbols lookup).

Those are only quick examples, one could write a book about that.

They were enough for me, but I agree it would always be nice if things were better.

@MichaelHale Oh no, don't you dare making me look picky :p I'm not talking about whole 4GB of docs but about couple of pages of essentials that should be perfectly clear for a system that is this old.

@Kuba I'm just saying I haven't even read most of the existing documentation on boxes to even determine if it is good or bad. What about guide/LowLevelNotebookStructure? tutorial/RepresentingTextualFormsByBoxes?

tutorial/ConvertingBetweenStringsBoxesAndExpressions

tutorial/LowLevelInputAndOutputRules

@MichaelHale tutorials are mostly fine. I don't complain on docs in general, but answer yourself how easy is it to learn the real main loop with understanding. Is it in one place in docs, is it at least explained in order somewhere? Symbols lookup explanation is inconsistent.

@Kuba But do you really think that starting out with a deep explanation of boxes and the main loop is a more streamlined way to start learning for most people than "here's how to type input and get output. here are some common functions and what they do. here are some pages which categorize other functions by their area of use, click on them to read more"?

@MichaelHale no, but after becoming fairly familiar with basics one may need to understand fundamentals

and it will not be possible for someone without IT background, or with limited amount of time or patience

Do you think it's possible for people with little time or patience to learn how to program beyond just copying examples that are almost identical to what they want to do?

I think the point is that it's not as well explained as it could be, I can sympathize with that. Although I have never needed that knowledge.

@MichaelHale I had limited time but a lot of patience and I think I've learned more than I "needed". But I wasted a lot of time because of those issues.

@C.E. this + parts which are not explained at all + parts that are explained in a really missleadig way

And again, I'm not talking about details but basics.

@Kuba I know that you are talking about basics it's just that I've never had the need to understand that part of the loop. It's obscure basic, not basic basic.

Agreed. But it is essential for everything more than "basic package".

But "basic package" covers a pretty broad range of things. Everything from curing cancer to inventing a better light bulb.

@MichaelHale no if you want to sell/share/hide to source. Or show an example of cdf to a potential investor? Is there a way to automatically embed a package in a cdf so that it is standalone and the guy who has no idea won't have to care about installing a package? I'm not talking about SaveDefinitions, one often needs more than to load saved values.

While basic package covers a lot you still need more to do something modern or combine different things MMA ofers you.

@MichaelHale I think its got potential as a GP language which is why I said what I did. Its that huge amount of functionality that is incredibly daunting though. Its that for which I think a more easily digestible 'subset' is needed to let people get their feet wet in as productive a manner possible.

@C.E. Hah, I didn't want to mention 'The Good Parts'. I don't know if there are 'bad' parts to 'Wolfram' yet. I'm still trying to wrap my head around what parts I actually need for the moment. I hated Javascript until that book came out. It identified the cruft and told me of the confusing parts. Ironically, later on, we spent some time using the 'bad' parts because they actually were useful in some very narrow edge cases.

@Kuba I did not know that there was a main loop that was accessible. i figured it existed yet since I had not seen any discussion of it I figured it was 'hidden'. Now I know I need to go look for information.

Out of this discussion, I've seen several items at which I need to look.

@LibertyTrooper being accessible is another thing :)

@Kuba Well, maybe it can be Spelunked? :-)

Showing it is possible but there are no functions like WhatKernelsSeesJustBeforeEvaluationForm

you really have to know what you are doing to show those things

No method to hook it? Supported or not?

e.g. FullForm with context for each symbol?

@Kuba Actually, I probably should back off of that... I'll probably just confuse myself.

Which will lead to another round of annoying questions in chat.

@Kuba My friend showed a CDF I made to investors. He says it worked. I haven't needed to do anything beyond SaveDefinitions yet.

@MichaelHale Great! What happens when they want an App for that? :-)

@LibertyTrooper This page isn't a good core set of language features?

reference.wolfram.com/language/guide/LanguageOverview.html

@LibertyTrooper There is a separate app for the data collection part. They demoed that too. The CDF was just the server side analysis part.

@MichaelHale One point that was brought up earlier is that Wolfram is 30 years old. Just like Javascript, I'm sure a great deal of cruft has accumulated. The documentation being so expansive is what makes it confusing I think.

@MichaelHale I don't claim it won't work. It won't be a general solution though.

@LibertyTrooper I completely support making breaking changes to the language. But that's probably because I don't have a huge code base I'm in charge of updating.

@MichaelHale Sorry, I was being glib. Yet usually I've seen 'POC's' rapidly morph far beyond what I originally intended...

@MichaelHale Maybe what I want is "Mathematica: The Good Parts".

@MichaelHale I think being backwards compatible is an advantage for both Mathematica and, to a lesser degree, Javascript.

@LibertyTrooper There is a neat summary of the main evaluation loop (the normal one, not boxes and stuff), on page 191 in Wagner's Power Programming in Mathematica.

@C.E. I'm wading through that. Its got a lot of good information in it. I've gotten some explanations out of there. Would like to see that book updated one day. On the day that I an take advantage of it.

Right now, its overkill for me.

I want to simply know the idiom that represents a try finally.

Or on error undo retry.

Once I get those and other basic idioms down I'll feel more confident.

Oh, that and how to manage a damn GUI. Not sure what to do. GUIKit is obsolete, right?

Spelunk is really nice.

@LibertyTrooper GeneralUtilities`PrintDefinitions is even nicer

Yep, GUIKit is obsolete

You need to go through tutorial/IntroductionToDynamic

BAD ASS

oops.. sorry.

and Advanced Dynamic Functionality

I wish someone had pointed PrintDefinitions out when I asked how M selects the import format...

Next you'll tell me there it will draw flowcharts too.

@LibertyTrooper Wouldn't have helped you, I commented on that question and I used PrintDefinitions to try to understand the precedence of the down values the day before. Didn't figure it out.

@LibertyTrooper I still think you are looking for something more complicated than you need. Do you just want this?

str = OpenRead["ExampleData/USConstitution.txt"];
x = 1/0;
Close@str

and

Check[
  str = OpenRead["ExampleData/USConstitution.txt"];
  x = 1/0,
  x = 0;
  ];
Close@str

@MichaelHale Oh, I got the e-mail import working in 2 lines of code.

In both of those the stream gets closed. Just like a finally. The second one allows you to do something else before closing when an exception occurs.

@MichaelHale Sorry, I misunderstood the pupose of the example.

So the system couldn't throw an error between the execution of check and Close?

Close is guaranteed to execute?

Barring a motherboard cookoff?

Not if you do something that crashes the kernel

But hopefully they fix those usually

So there isn't an idiom that guarantees to execute clean up no matter what.

I guess in such a high level language it doesn't matter.

They could call abort within Check, but there is also CheckAbort. I've never needed to use that though.

Ok. So I won't spin my wheels looking for a guaranteed mechanism.

Or an exeption chain.

Well I mean even in C# if you kill a program with the task manager it won't execute a finally block is my understanding.

KILL vs. HUP

Thanks for the insightful conversation. I have to run.