@rm-rf Good morning. I guess you weren't in bed since we talked :-)

@halirutan Just about 6 in the evening now :)

About 3 in morning here, but I went to bed at 8..

Ok, here is the thing:

I had/have my own installer routing which basically does the same as Leonids, but isn't as general as his installer.

Since I now was splitting a small sub-package from another one to make this publicly available, I thought of joining the development of Leonids installer, because I'm kind of sick about, that everyone is hacking their own stuff while we could easily join forces here.

I agree. There's also Rolf's mini installer

After trying it, I saw that it doesn't fit my needs in its current form and therefore, I was talking to Leonid about this plans. My biggest concern currently is, that the Installer only supports the easy package layout while all of my projects have the structure suggested by WB.

Furthermore, since many public projects are hosted on GitHub, I would like to have a way to first check whether or not a package needs an upgrade. On GitHub we not only have access to the final master.zip, but we could check any standardized version file like Leonid already started in creating the project.m files.

@halirutan Except, not everyone uses the project.m files (most leave it there because WB puts it there) and some times, the top branch is not even master

@rm-rf Yes, I don't use them, but for the sake of version check in an upgrade I would create one.

Maybe there is a better way.

(sorry, I meant Pacletinfo.m... either way, my point stands)

@halirutan If it's github, you can just compare the commit shas

@rm-rf But they not necessarily mean a new and better version.

@halirutan No, but it is common practice to not push a breaking update to master/top branch

I'm not following the Git Flow Model throughout all my projects, where in the master branch is always a final version.

@halirutan Hmm... Ok. I try to do it that way (except for private repos, which are mostly for backup purposes)

But yes, barring that, an "update" file of some sort would be prudent

@rm-rf Another thing: it is not clear to me why the Installer destroys file permissions.

Basically, if they are kept in the zip file, then unpacking is done by Java and Mathematica only copies the stuff. For this it should rely on the OS anyway.

@rm-rf Currently my biggest concern is that, although I know what Leonid has planned, I'm not sure how he wants new features be implemented.

@halirutan in fact, WB's "deploy" doesn't preserve permissions either. I don't know why this happens either, since I too thought that the OS handles it.

I don't want that he has to rewrite all of my code because I didn't got the concept he was trying to implement.

@rm-rf When you download the master.zip the permissions are OK, right?

@halirutan Yes

@rm-rf OK, then this can be tracked down easily. Maybe I should start with this.

@halirutan I tried looking at this bug last year, but it seems like Leonid implemented much of the functionality in Java, so I was completely lost there.

@rm-rf Yes, I would start at the Java side and write an unpack test.

For a github test repo, you can use MATLink. It has some executables in it and the x flag must remain after extraction.

@rm-rf Yes, that was the plan.

@rm-rf This is going to be a bitchy one..

Uh oh.

@rm-rf The problem is that Leonid goes through the list of files in the zip, creates a new one (or a directory) and copies the bytes in it.

(This has nothing to do with Leonid, it's common practice)

The easiest solution would be to use setPosixFilePermissions(). The problem: It was introduced in Java 7 but Mathematica ships only with Java 6

And now you can read through some SO answers and find things like

> In earlier versions of Java, using native code of your own, or exec-ing command-line utilities are common approaches.

And this was an answer of a user with 115K rep

hmm.

@rm-rf And no one wants to implement OS calls to change the file permissions for all 3 operating systems.

certainly not

@rm-rf OK, I think a make a search whether someone clever has written a library for that.

@rm-rf In the next release we will have Java 7.

@halirutan Is there a need to use Java? CopyFile keeps the permissions...

@rm-rf how do we extract zip, tar, tar.gz?

@rm-rf Although we have ExtractArchive (I haven't checked whether this preserves permissions), this was introduced in 8. So all users < V8 are screwed.

With[{zip = "/tmp/matlink.zip", out = "/tmp/MATLink"},
 URLSave["https://github.com/rsmenon/MATLink/archive/master.zip", zip];
 If[! DirectoryQ@out, CreateDirectory@out];
 ExtractArchive[zip, out];
 ]

That was the reason for me too to write a java unpacker.

@rm-rf Yep. When I started to write my downloader, it was shortly before I got the Pre8 version where I saw that it could all be done in Mathematica

@halirutan That's definitely a decision to be taken and I would favour having v8 (or maybe even v9) as the minimum. Look at mma.se... very few people using v7

It preserves permissions

@rm-rf Then we should this suggest to @leonidshifrin

This discussion is amazing ... so so excited for this

Hi @rm-rf, @halirutan. So, what would you guys think is the biggest problem for the installer now - file permissions? As I tried to explain in the comment to the layout issue opened by @halirutan, the nested layout created by the WB is not the one that should be used for deployment. Anyway, I have removed that check, so now any layout should work.

Also, with the new functionality for ExtractArchive, it might be that my Java code is altogether unnecessary. I have to admit that I stole the unpacking code from the web, fixing a couple of bugs there. Implementing OS calls to change permissions would be Ok, but certainly more work. May be, ExtractArchive preserves the permissions, I just have to check that.

@LeonidShifrin The permissions thing might not be a problem for most projects, but I think it's something that needs to be fixed at some point because losing permissions just seems "wrong". ExtractArchive does seem to preserve permissions.

Secondly, I'm not certain you should be installing to $UBD/Applications if the structure resembles that of a WB project because we don't know how the developer intended for it to be deployed. I think your original check was useful and should be retained.

@rm-rf All right, I will consider moving to ExtarctArchive to be the top item on the todo list. Also, as I looked at the code a few days ago, it needs some refactoring, since I wasn't able to immediately see what was going on there, which isn't good.

@rm-rf Re: installation check: I was also thinking like that before, but imposing rigid check might also be an overkill. I think best would be to keep it as an option which is enabled by default but can be disabled. I removed it as a quick fix for @halirutan to be able to use it with his layout, although, as I mentioned, it seems to me that that layout isn't valid.

@rm-rf It will be hard to allocate any time for this in the next few days, but I may be able to do so closer to the next weekend.