last day (17 days later) » 

hBy2Py
hBy2Py
02:41
Note to self, cull this comment:
Geoff, when you have a moment, could you join me in a custom chat? I have some questions about OpenBabel / Avogadro development. Thanks! — hBy2Py 21 secs ago
Geoff, thank you in advance for your time.
I'm developing h5cube, a Python tool for CUBE file compression via h5py/HDF5.
Per pentavalentcarbon's suggestion, I posted notice to the avogadro dev email list a while back, but didn't get any responses.
As a side effect of putting the package together, I've effectively defined an HDF5 file specification for these compressed CUBEs.
(I plan to write up the specification more comprehensively as part of the pending product documentation.)
In any event, this seems like something that could pretty readily be incorporated into OpenBabel / Avogadro.
Lossless compression tends to reduce filesizes by ~7-10x
If lossy compression is acceptable, say when only a specific isosurface is of interest (e.g., when for archiving data in ESI for a publication), I've seen size reductions on the order of 200x or more.
So.
My C++ is pretty rusty, but I figure I could probably make a .cpp for the format to wrap into OpenBabel.
The big question on my mind is, are there any license problems with either statically or dynamically linking the HDF5 library?
Geoff Hutchison
Geoff Hutchison
OB won't want to link to HDF5 - it's too big of a pre-req.
But..
Avogadro2 already uses HDF5
hBy2Py
hBy2Py
@GeoffHutchison Oh, no kidding?
Geoff Hutchison
Geoff Hutchison
github.com/openchemistry/avogadrolibs
hBy2Py
hBy2Py
Ahhh, I'm late to that party, then.
Am I duplicating effort with h5cube?
Geoff Hutchison
Geoff Hutchison
No, no.
I think ways of providing compressed cubes are great.
Particularly if you can beat cube.gz or cube.bz2.
hBy2Py
hBy2Py
02:54
I have no experience with them... I'll have to look into what they can do.
They probably win with lossless.
Geoff Hutchison
Geoff Hutchison
I'm working on some improvements in Avo2 that would make it easier to call Python code, e.g., in io
Yeah, I'd check to see how gzip and bzip2 compare first
Since it's easy to get those "for free," particularly over the web
hBy2Py
hBy2Py
Mmk, so.. on that I/O, Avo2 would be looking for, what... a text stream with the CUBE contents? Or would it take binary input?
Geoff Hutchison
Geoff Hutchison
But if you have ways of semi-lossy compression in a cube, that's interesting for some applications
Avo2 has a way that Python scripts can be set up as file filters for particular extensions.
hBy2Py
hBy2Py
Yep. You can either truncate precision (mantissa digits) or threshold values.
Geoff Hutchison
Geoff Hutchison
So you could write a script that says "I'll take .zyx"
hBy2Py
hBy2Py
02:57
I'm using the builtin(?) HDF5 gzip, so identical values get squashed way down.
@GeoffHutchison Haha, oh, you mean just direct gzip or bzip2 of the CUBEs, got it.
Geoff Hutchison
Geoff Hutchison
Yes, I remember the post. Sorry I didn't reply - I admit I'm getting through severe email overload.
Yes.
Most people gzip or bzip2 the cubes.
hBy2Py
hBy2Py
I was Googling for "cube.gz" -.-
oy
Geoff Hutchison
Geoff Hutchison
So you'll need to show there's a decent improvement over the .cube file.
hBy2Py
hBy2Py
<nod>
No worries about not responding, I'm not upset
Figured I'd mention it in case you remembered it.
Geoff Hutchison
Geoff Hutchison
I appreciate that.
hBy2Py
hBy2Py
02:59
Where can I find what I need to link it into Avo2?
Geoff Hutchison
Geoff Hutchison
Hmm. You'd probably need to look at the hdf5dataformat link I sent.
Marcus Hanwell would be the most likely person to ask (on the e-mail list)
hBy2Py
hBy2Py
Looks like here?
github.com/OpenChemistry/avogadrolibs/blob/master/avogadro/io/…
Geoff Hutchison
Geoff Hutchison
I'm not 100% sure whether you'd be modifying that part or something in the separate quantum IO parts.
Yeah, there's also this:
github.com/OpenChemistry/avogadrolibs/tree/master/avogadro/…
My guess is the easiest is to add an hdf5 portion into the quantumio directory.
hBy2Py
hBy2Py
Can I assume the HDF5 library to be available at compile-time?
Geoff Hutchison
Geoff Hutchison
Yes. Look at the linking parts in here:
github.com/OpenChemistry/avogadrolibs/blob/master/avogadro/io/…
if(USE_HDF5)
list(APPEND HEADERS hdf5dataformat.h)
list(APPEND SOURCES hdf5dataformat.cpp)
endif()
I need to get some sleep, but ping me either here or by e-mail.
Sounds pretty cool, thanks!
hBy2Py
hBy2Py
03:05
<nod>, sleep well.
Thanks for replying!
room topic changed to Avogadro, OpenBabel and h5cube: Discussion of integrating h5cube with Avogadro (no tags)
room topic changed to Avogadro, OpenBabel and h5cube: Discussion of integrating h5cube with Avogadro [computational-chemistry] [software,]
room topic changed to Avogadro, OpenBabel and h5cube: Discussion of integrating h5cube with Avogadro [computational-chemistry] [software]
 
11 hours later…
hBy2Py
hBy2Py
14:09
@Geoff Initial benchmarking on a 250x250x250 electron density CUBE of benzene is complete.
Performance appears favorable.
40% faster than gzip for 80% better compression @ fully lossless.
80% slower than bzip2, but still about 40% better compression at fully lossless.
I'll put together some isosurface plots (in Avo, of course :-) of these various compression levels, but based on earlier playing, I expect everything but the -t 1 isosurfaces to look basically identical.
So, if all one needs the CUBE for is plotting a single isosurface, that 180x compression factor should be representative.

  last day (17 days later) »