last day (15 days later) » 

01:51
-1
Q: RAM overhead on a compression program

thexivIn a program, the one below is attempting to codify into a translation of another file. The output is severely condensed. It works, and once I get it to decompress, then it will win award after award. However, in the first I must make accommodations for computers with smaller memory. Just before...

"If it works, and I get it to decompress, …" — I am skeptical that this should be considered "working" code, given that introduction.
This is working code. I will edit that out. you can test it yourself. There are no errors. However, the code itself is so impractically magical that it surprises me the data is there.
It works because I have a 3 node binary tree, and I'm saving them parallel in three formats of 64 bit bytes. The page vector is to save the key points where the hashing could not be gotten back without manipulating the bits further than <<>> 1-2 times. I get free bytes from the hashing at points, and I have that in pct. I figure in a future version, I may have more worked out.
If you run this code, it will work smoothly, surely, slowly. However you will knock at least 85% off your entire file. Syntax: bincom.exe input.file output.file
@thexiv I just compiled your code and ran it against two test files. It seg-faulted. Are you sure this code actually runs against any arbitrary data file? What file are you using to test with?
I used the hutter prize files. The 100 mb file went to 9 mb, and the other, the 35 when to 3.4. I also used the c++.exe file
What kind of files were you using? And what OS?
@thexiv I've looked up the hutter prize file, and I'll attempt that both with the current build (mingw64) and a Microsoft build (VS2015). I'm currently running Windows 10 64-bit.
01:51
I am too. But I'm using cygwin
@thexiv Regardless of if this runs or not, I have a ton of suggestions for you in the form of a code review, if you want them.
Yea! I'll take anything I can get!
By the way, my file takes a very long time, I don't know if you caught that in the comments. It's slow but man what a compression job it does. @phyrfox
Mat
Mat
Code as posted doesn't even compile. Fixing the obvious errors leads to an executable that segfaults on every input I tried...
If you use cygwin64, and you have Windows 10, compiling with the c++.exe command with the option of -std=c++14, then you will have a working program. Other than that, I cannot assume the problem. But I'll take advice.
Mat
Mat
01:51
Still segfaults on empty file, or file filled with zeros.
hmmm... Okay, I should idiot proof it then. I know why, it's the do...while
Mat
Mat
Now always segfaults but produces output..... that is bad. A file with 4096 "a" in it compresses to the same as a file with 4096 "b" - decompression impossible (and compression ratio worse than plain zip).
That's not the way I get it. I don't know why your system is doing that. Are you using cygwin64? What OS?
@Mat Sorry I didn't quote your name in that. So all the innards of the file are the same? That doesn't make sense since I'm getting all predictable outcome from my end.
Whoa. I just noticed the edit history: you have edited the hell out of the code in this post. It appears that you didn't originally post working code. Please, don't edit the code in questions. People can't answer, and refer to your code, if the code keeps changing.
Just above the line that says h2 = (h1 ^ y[1]);, insert a line that says assert(y.size() >= 2);which must be true if y[1] exists. It doesn't and the program will fail on the assert. This code is broken.
01:51
It should have that yes. I will add it to the code. Thank you. I'll also have to add that I need to have the last one be y[1].reset() in case I run into an odd number of n-2 tiers in my pyramid.
02:08
Sorry for all the dysfunctional talk. I hate to just be bombarded with no, no no no. It does work, on the copy I have. I get 89% compression. It's because I'm testing for a lot of similarities. The program itself is only going to increase in size because of the directing it does to provide instances where the hashed final constant is made true. This state allows me to unbundle the program. I did fix the memory problem. Now, it's a dream. Takes an hour to compress 98 megs to 19 megs.
But I am sorry. It works for the better half and it's solving low, so I'm excited. There's also a big contest for this and I want to be a winner in it. So the prize is kind of disjointing me from anything but the decompressor.
 
13 hours later…
14:46
@thexiv: Would you like some help with this?
In order to post a good question, you need to assure that your code is ready for review.
One way to do that is to do thorough testing of your code before posting it.
Another useful technique is by putting in asserts such as the one I mentioned above to catch any condition that "shouldn't ever happen".
To help with portability problems, you can try the code on another platform if one is available to you. One such place to do so would be an online compiler such as coliru.stacked-crooked.com
Finally, try some sample inputs. For example, try a file with 1000 repetitions of a single letter and report both the input and output results (not the contents necessarily, but at least the file sizes) if you should decide to re-post the question.
Also be aware that the system will auto-ban any user who posts too many questions that are closed as off-topic, so try to make sure that your future questions don't get closed by adhering closely to the help center guidelines
Good luck!
 
1 hour later…
16:27
Hi mat, sorry for my coding. I ended up telling the program not to use it unless it's more than 50 bytes. That solves for it because the program differentiates past 4 bytes.
@Mat'sMug
hey there
so, did you read any of the help center links we've been giving you?
a little, i understand I should not continue with new questions in the same space as an old one with an answer
I think that was my problem there.
and then also, I thought this site was for reviews
that's because we value reviewers' contributions - people spend a fair amount of time reviewing code (often >1h per answer), it's simply not fair to edit the OP on them and invalidate their points with a chameleon post that updates every time someone points something out - so that's one thing.
16:32
I've always fought reading those guidelines because I just convinced myself that, A. it'd be okay to try, and B. that it'd be hard to read.
yea I apologize
i understand that
My thought on that was that, I fixed the code, and I should put it up. But obviously that was wrong.
I meant to finish that previous thought. I'll read them from now on
TBH everything went wrong here. the normal course of action would have been:
I can do it
- you post broken code
- question gets put on hold to prevent answers while you fix your code
- you update the question with the tested and working code
- question gets reopened
- people review your code and make suggestions on what to improve and how and why
- everybody's happy
we apparently failed at the 2nd step, and an answer slipped through
16:37
oh, well I fixed it, then I updated it, and I was waiting for reviews
should I take this to Programmer's Stack Exchange?
nope. way "too localized" for P.SE
now, you have working code?
yes
All different chars coming out instead of the 'y' with the marks above it
practically random looking
@Edward was saying your last revision (the one I rolled back) still had segfault issues (I don't do C++) - if your code isn't ready for a peer review, I'm afraid posting a new question with it isn't going to end up nicely - we'd rather avoid that.
but it does "tell a story" of how to put the file back in order
right
the thing with that is, I don't get seg faults
I don't know why everyone is having trouble with it
I have the most up to date cygwin64
I'm only using the standard compiler option for c++14 and I'm using c++.exe
if it works as intended in all cases you need it to work for, to the best of your knowledge, and you're not asking anyone to fix any specific code issue (e.g. segfaults), then I don't see it getting slammed-shut.
16:43
well this one didn't get slammed shut, that was posted 3 days ago
but i see where your coming from
it seems to work with the g++ compiler too
even faster!
cool. the idea behind code review is basically "does this code make my ass look fat?"
yes :)
i was looking to destroy the memory problem I had
and I answered that on my own
now if I could just store away all these cauterized images in my head..
I got my current code to go to 10 percent of the original file
usually posts containing "problem", "bug", "issue", or with a title that looks like a question (e.g. "why does..?", "how do I..?", etc.) aren't good fits for Code Review
ahh okay
so does the memory problem count as a inherent "issue" question, thus off-topic?
TBH it's gray-area, that's probably why it wasn't "slammed" 3 days ago ;-)
16:47
I got 3 favorites on it
that was a fisrt
first
I'm trying to find someone to help me code the decompressor
that won't fly anywhere on SE though
yea i know
ok
so, if you put your working version up for review in a new post, with a nice title that roughly says what problem the code is solving (i.e. what it does), and a decent description of it in the post body, with the whole working code in its glorious context (we love context/surrounding code!), I'm sure you'll get valuable feedback and a massively positive experience on CR (as usually happens)
okay
thank you for your help sir
no problem

last day (15 days later) »