@ChrisK and anyone else with interest, here's the current state of things:

1) I have a stylesheet which has an Editing style environment which is how you change things:

It looks kinda like a normal docs notebook. You can insert templates, save things, edit the metadata, etc. Easy to extend if I want to later. The menu in top right lets you save the thing to a paclet that you give by name.

2) After a RebuildPacletData you can search for the function (the name I gave the docs notebook was Test):

So everything works normally there. This opened notebook isn't save-able so people can't corrupt it without trying.

3) You can export the Markdown file. I add in <a>...</a> links in general because they let me provide #-based search in the browser. But we can see how things look in the most basic Markdown->HTML case:

Now I just need to add a few usability helpers and a theme to Ems so that you can get really nice looking docs from this stuff.

One really nice thing (for me) is that this will be much faster to work with than the stuff I had been doing for my documentation auto-generator.

@b3m2a1 👍

I need some new idea here..

Help

@yode you've made it very hard for us to help you... you should gather a representative set of images such that if we can solve the problem for that set of images, then the problem is solved.

because algorithms in image processing typically will have parameters, they will have thresholds, and they will not be universal. This is fine; what you have to do is analyze your set of images to figure out how to separate the areas you are interested in, in such a way that it works for all images.

@b3m2a1 it looks good, but please don't feel constrained by Markdown. It's OK to export divs and other things with classes and stuff, if you need that to make it look good. In fact, I don't really see why not just export it directly to the HTML template that you have in mind for the documentation.

@C.E. I think the Markdown can actually do well enough, as long as we put some work in on the CSS. I like the idea of the Markdown as it increases the possible reach of the docs. Everything can process Markdown. It allows people to use Jekyll, or pelican, or any number of other site generators, rather than being constrained by the HTML export I choose to do.

I'm pretty sure I can be smart about the CSS and get the final HTML to look good, too.

@C.E. Done..

@yode nice, I will take a look at it this evening when I come home.

@C.E. Thk.. :)

I think must have some pretty method can do this. Such as we can consider the pixel as a stuff with weight in the negative color image. But I just don't know how to implement it..

@yode I convolution on the negated image with a box kernel would probably do pretty well, something like that. But I can't look at it right now.

You mean tmplate match or similar something?

@yode it's that kind of solution, yes. I think it's promising. The template doesn't have to be perfect. Using a box matrix as a template will suppress the lines and highlight the gray box areas.

Then in the next step, one has to consider what one can do with that image. But I think it's a step in the right direction.

In this negate image:

If I think every white pixel is a stuff with quality...

I will check it when I get home.

Then the mark area will more get stuck into it..

Ok..

@ChrisK @C.E. okay I've put in what I have time for today. Now on top of the ability to make these notebooks and have them naturally integrate into the documentation center (or a paclet) I now have a partially complete theme and set of templates for building a site for these docs from the generated Markdown.

The theme is definitely not done (I need to clean out some things, may stuff more attractive, etc.) but it's much nicer to work with than the standard WRI export docs notebook to HTML system.

And the search functionality is just tipuesearch so it works really nicely and intuitively.

What tools and features do I need to make it so that it's really easy for people to use this?

I need to dump some intermediate results, because there is not enough memory, and then read some of the specified lines, is there any good solution?

But compression rate is much lower than ByteArray

@b3m2a1 I would like the option to post code as copyable images, like in the WRI online docs. It would also be nice to be able to fold and unfold sections.

re. usability, I'm not knowledgeable enough about how it works at the moment to comment on that.

@b3m2a1 I did not notice any change, can you email me with details?

@b3m2a1 Something like this would be great. The problem is that I would only feel safe to invest in any documentation technology, if I feel that it is well tested, works with 10.0+ (to produce documentation that also works in 10.0+), and most importantly: it will be maintained for years to come.

It is very difficult for a single person to commit to a project like this.

What I said above applies to any system that works within Mathematica and integrates with the documentation center.

For that part that converts existing notebooks to nice Markdown or HTML, I am definitely interested, as this is something that can already be useful in the short term. If it breaks at some point, I could stop generating HTML pages, and stick to notebooks.

However, it it forces me to make significant and irreversible changes to the source notebooks, then we're back to what I said above.

Was anyone watching the Twitch streams on Standalone Applications, and if yes, could you give a quick summary of what standalone means? Standalone as in able to run without having M installed?

A quick skim seems to suggest that this is not the case.

@Szabolcs My understanding was that they were going to bundle Wolfram Engine with .wls scripts. The Wolfram Engine will have a digital signature that says that this particular Wolfram Engine can only run this particular script. You will get the digital signature by registering your app on a website. It's about 43 minutes into the meeting.

The script can launch a frontend e.g. with Manipulate or whatever. There will be different styles available for this frontend.

@C.E. Which meeting is that? The recent one or a previous one about Standalone Applications?

Did they say how big the wolfram engine was?

@Szabolcs Not that I know of, they talk about it here (from there and two or three minutes.)

Yes, I found it.

I'm quite sceptical ... bundles would be huge. The system is purposefully non-modular, so they can't even cut out the unneeded parts easily.

Mathematica is not good at creating robust applications, even if there is only a command line interface.

BTW for command line: the startup time is horrendous, so I do not see people running this repeatedly in a terminal.

@Szabolcs I have the exact same reservations.

For scripts it may not make sense, but for apps it might. Maybe you want to write an app to control your lab equipment for example. In that case neither size nor startup time matters.

Imagine a non-Mathematica user downloading their 10th such application. By that time they will fully expect a 10-minute download time for something that does only a fairly trivial task with a very laggy UI that randomly falls apart in a mess of pink boxes ...

They probably just won't download the 11th.

Just to be clear: I do not think that the Dynamic stuff is bad. On the contrary: it's extremely useful for personal work. I just do not think that it is suitable for creating robust and fool-proof UIs. Nor should it be: there are always compromises between use cases.

@Szabolcs @C.E. I sent you a message on Slack about this docs stuff, as that's a bit easier for me to work with than chat here. @ChrisK if you want in let me know and I think I can just add you to the thread or whatever.

@yode looks like Niki already has posted a good solution. I can't do better than that.