The DMZ

TildalWave

12:08 AM

Scottish is considered spicy?

actually yeah I guess it would be, I thought you meant hot

Rory Alsop

@TildalWave no - it was a joke

TildalWave

well you got me there because I honestly dunno

I love spicy hot food tho

I'll probably have my personal record number of chili plants and varieties this year, I counted 19 different species if all seedlings survive

jeez, it's Friday isn't it? shit I not only lost sense of time but of calendar as well

Ohnana

12:27 AM

@RoryAlsop thai is good shit

silverpenguin

12:48 AM

Waiting for someone to ask if we are talking about drugs from other countries and where we have to quote my first curry post

SmokeDispenser

1:08 AM

drugs?

I heard drugs.

;)

good evening guys:)

TildalWave

1:22 AM

3

Q: Is posting identical multiple, but reasonable, answers a spam?

Just reviewed these three answers posted one after another: http://security.stackexchange.com/a/117940/86652 http://security.stackexchange.com/a/117941/86652 http://security.stackexchange.com/a/117942/86652 I wouldn't consider the first one (or even every single one on its own) a spam, but po...

discussion

besides the issue in the question, I think the bigger issue identified there is that all those questions should be closed as off-topic / product recommendation

SmokeDispenser

You might be right. Yet, that can be kind of a blurred line depending on the exact wording. One of the questions does have 11 upvotes.

that might be a sign that users find that recommendation request reasonable.

TildalWave

@SmokeDispenser that question is 4 and a half years old, posted on a less than a year old and I think then still beta site

SmokeDispenser

@TildalWave, alright, it's too late. I missed that:)

TildalWave

oh no pressure, it's late here too

SmokeDispenser

then it might be best to just close it. it will not get roomba'd and keep those answers away.

Mark Buffalo

1:34 AM

Anyone here know anything about hex/bytes?

SmokeDispenser

@MarkBuffalo say what now?

Mark Buffalo

Trying to find the best way to handle conversion d:

@SmokeDispenser So I'm writing a hex editor from the ground up. I've got the file hex values down to a science...

But when converting those hex values to strings, I encounter issues. In particular, there are a bunch of characters which don't appear in a regular hex editor... those are usually replaced with periods.

Logically, it seems like I should convert all non-ASCII characters to periods, but I'm not quite sure.

TildalWave

@MarkBuffalo why are you writing that? there's loads of them, some even reasonably good

Mark Buffalo

@TildalWave because challenge

I need to spruce up my github with non-NDA stuff.

TildalWave

can't you find some normal challenge for guys that are normal?

like how fast can you down a pint of beer

Mark Buffalo

1:37 AM

But I'm not normal d:

SmokeDispenser

@MarkBuffalo, non-ascii? isn't ascii covering all of a byte?

Mark Buffalo

@SmokeDispenser Well, I can get the characters... but it just seems weird.

I'm not sure if I should split by sections of 16 or 14

@ThomasPornin Any insight here?

It seems the max is 16, but a lot of editors are using 14... which baffles me.

SmokeDispenser

@MarkBuffalo the characters may be non-printable. those should get displayed differently (like dots:))

Mark Buffalo

@SmokeDispenser Yeah, I think so. When using a StringBuilder class, I'm getting nothing out of it... except a couple letters. When printing them out in a ghetto way, I get all of them, but they're... they are ugly.

ok, gonna look into that. thanks

Looks like I can do something like:

private bool CanPrint(char c)
{
return Char.IsControl(c);
}

SmokeDispenser

@MarkBuffalo, if you use UTF8 in the output, there might be code points for symbols representing non-printable ascii characters?

that'd be neat - to see what that is, instead of a dot.

(without having to look in the table)

Mark Buffalo

1:44 AM

what the fuck

All of my google searches are being redirected to a strange server

They're being XSS'd

SmokeDispenser

Hu?

like, those from the browser or those on google?

Mark Buffalo

Yeah, Google searches are being XSS'd

luls

SmokeDispenser

you are using https, I suppose, that should make injecting js into googles page.. hard.

Mark Buffalo

Yeah, so that's weird

SmokeDispenser

maybe google got pwnd?

Mark Buffalo

1:49 AM

or maybe I did

SmokeDispenser

isn't currently the pwn2own running?:)

Mark Buffalo

and I'm not really connecting to google

MAYBE YOU'RE ALL FAKE

WTF

SmokeDispenser

@MarkBuffalo, we can check

Mark Buffalo

MAYBE I'M FAKE

SmokeDispenser

@MarkBuffalo INORITE!!1111

(actually, on my keyboard layout the 1 in not the same key as '!';))

@MarkBuffalo, let me double check google is fine.

@MarkBuffalo what's the sha of googles main page for you?

awww, damn, it does redirect me to the german site, even with curl.

Mark Buffalo

1:52 AM

you're getting owned by google?

Here's the SHA 256 fingerprint: 65:42:50:5F:5A:00:13:81:1E:88:E3:8A:2E:2C:48:4B:62:38:F0:F1:4A:6A:BB:49:45:BD:4C‌:06:BD:4D:D6:89

SmokeDispenser

of what exacty? I guess the german site is different from the us one.

Mark Buffalo

google.com

SmokeDispenser

curl google.de | shasum

fd2a79fe9ad73400e3115d15465d985cd800b751

I have no choice than using the de version, it redirects me... wait.

VPN :)

Mark Buffalo

> curl google.com | shasum
> 2b4a1edfff3170948dec9cee38106c661a48abdd

SmokeDispenser

FUCKUP:~ jg$ curl https://www.google.com
curl: (7) Couldn't connect to server

well.

Mark Buffalo

1:57 AM

luls

SmokeDispenser

I cannot help you -,-

<H1>302 Moved</H1>
The document has moved
<A HREF="https://www.google.de/?gfe_rd=cr&ei=UbLsVvuuOeyA8QfgjYP4CA">here</A>.

Mark Buffalo

heh

SmokeDispenser

what is that 4101 under my name?

isn't that supposed to be rep?

Mark Buffalo

The total number of reputation you have site-wide

Ya

SmokeDispenser

ah, it's counting SO

I C :)

Mark Buffalo

2:02 AM

Okay, return Char.IsControl(c); didn't work right.. hmm.. another step is required...

SmokeDispenser

20 mins ago, by SmokeDispenser

@MarkBuffalo, if you use UTF8 in the output, there might be code points for symbols representing non-printable ascii characters?

If you did that, you'd have to hard code all of those. but that'd be neat.

Mark Buffalo

no, you can use a regex range instead of hard-coding

SmokeDispenser

well, depanding on the code points yes

but I'd guess you'd have to hary code the relation.

Mark Buffalo

There's a lot of unicode slashing going on here

SmokeDispenser

btw, can I ask you a favor?

I cannot see the up/down count yet here.

Mark Buffalo

2:07 AM

Link me

I don't know if I can either

SmokeDispenser

can you check the up/down on a answer of mine? I got +8 like 2 minutes after the rep cap passed.

that would be 1 up, 1 down, but in 5 minutes in the middle of the night?

Mark Buffalo

LINK IT

SmokeDispenser

here

Mark Buffalo

LINK IT OR I WILL CRUSH YOU!!

SmokeDispenser

@MarkBuffalo, ON IT•DONE IT•DAMNIT

Mark Buffalo

2:09 AM

Oh wow... all this time, I didn't know I could view the totals by clicking on the number

SmokeDispenser

haha, i like my layout when screaming;)

Mark Buffalo

Yup, one downvote.

SmokeDispenser

allright, thanks:)

Mark Buffalo

Hang on, I'll give you another downvote.

j/k

SmokeDispenser

:D

the rep cap is a funny thing.

without it, I would have been able to check that myself;)

Mark Buffalo

2:13 AM

fortunately I repcapped today

but couldn't really participate

SmokeDispenser

since i moved here from SO i repcaped every day (well, only 3 days)

with a top answer of 45 (46 up/1 down, as you tell me) on a hot network question that nettet 38 rep (now, after the cap lifted)

on SO, I never hit the cap.

Mark Buffalo

Questions and Answers are higher quality here... I rarely need SO anyway

SmokeDispenser

me neither, especially since I'm starting a new job (hopefully) soon that'd make security my new focus:)

Mark Buffalo

Nice

Where at?

SmokeDispenser

BSI

wait

linked it.

@MarkBuffalo, did you figure out the XSS?

Mark Buffalo

2:26 AM

no, and don't care.

SmokeDispenser

haha

Mark Buffalo

this is a derpy console

SmokeDispenser

so... it's late here.

I'm gonna get some sleep.

Mark Buffalo

later man

SmokeDispenser

later:)

BTW, no comment on the new place?;)

Thomas Pornin

2:29 AM

@MarkBuffalo I don't follow the weirdness of your discourse

Tools that display binary data into hexadecimal, like Linux's "hd", use lines of 16 bytes, in two groups of 8.

Mark Buffalo

@ThomasPornin Me neither. So, you know when you display the hex values on the left side?

I got that part.

On the left side, I don't understand why there are so many periods, instead of what those characters actually represent

or if I'm supposed to use UTF8, Unicode, or ASCII...

SmokeDispenser

@MarkBuffalo, because they are not printable, as I said:)

Thomas Pornin

Do you really understand the differences between ASCII, latin1, code points, code units, and UTF-16 ?

Mark Buffalo

That's what I thought

@ThomasPornin A few things about it, but not fully

SmokeDispenser

and being able to switch between ascii and utf8 interpretation would be neat;)

Mark Buffalo

2:32 AM

I am hoping you can point me in the right direction

Thomas Pornin

Unicode defines code points. A code point is a number in the 0 to ox10FFFF range (fits on 22 bits).

Every glyph in Unicode corresponds to a code point.

Mark Buffalo

Yeah, I understand this part

Just not the terminology you mentioned

...until now

Thomas Pornin

A code unit is the building block for encoding a code point. UTF-8 uses bytes as code units. UTF-16 uses pairs of bytes as code units.

SmokeDispenser

good night then, later. have fun hexing:)

Mark Buffalo

@ThomasPornin ah, now I see...

Thomas Pornin

2:34 AM

ASCII is an encoding that defines code points 0 to 127, and uses bytes as code units, with the simple "1 code point" -> "1 byte" mapping.

Mark Buffalo

or now I see part of it

Thomas Pornin

Latin1 (ISO 8859-1) is an extension of ASCII that goes from 0 to 255, still with a simple mapping to bytes.

Mark Buffalo

Ya

Thomas Pornin

UTF-8 maps each code point to 1, 2, 3 or 4 bytes, depending on the value.

Characters with codes 0 to 127 go to one byte with that exact value, so anything encoded in ASCII happens to be also valid UTF-8 encoding of the same.

The same does not hold with latin-1.

Mark Buffalo

@ThomasPornin mh-nexus.de/en/graphics/HxDShotLarge.png

Note on the right corner, NTFS

Thomas Pornin

2:37 AM

For instance, the code point 0xE9, for "é", is encoded in latin-1 as a single byte of value 0xE9, but in UTF-8 as two bytes 0xC3 then 0xA9.

Mark Buffalo

ah

Now, ASCII encoding will not handle this kind of output, but UTF-8 does

What I want to understand are the "dots"

It seems all instances of "00" map to dots

Thomas Pornin

The code point "0" corresponds to a non-printable character.

Codes 0 to 31 are non-printable; they are often called "control characters".

It is customary for hex editors to render non-printable characters with dots

Mark Buffalo

So I should simply check to see if the codes are anywhere from 00 to 31? Or use Char.IsControl(char) to detect it?

I should check this now... since after our conversation, I got it working using UTF8

Thomas Pornin

You keep talking of "UTF-8"

which should have no relation whatsoever with the problem at hand

Mark Buffalo

Well, ASCII encoding was not showing all text results. I'm probably doing it wrong.

Thomas Pornin

2:42 AM

There are several ways in which "text" can be encoded into bytes. An hex editor cannot show them all at once.

Mark Buffalo

@ThomasPornin I encoded the hex string to bytes, and then used Encoding.UTF8.GetString(charBytes)

Thomas Pornin

Hex editors usually follow ASCII or latin-1: for each byte of value x, they show either the character whose code point is x, or a dot if that character is not printable.

Mark Buffalo

I have a lot to learn d:

Thomas Pornin

@MarkBuffalo Down that path lies only sorrow, tears and grinding of teeth.

A sequence of bytes may be a valid UTF-8 encoding of a sequence of code points. It may also not be.

What happens in the latter case with Microsoft's code is anyone's guess, but it is usually sloppy.

Mark Buffalo

Are you referring to Encoding.Format.GetString(bytes)?

So I should probably do this manually?

Thomas Pornin

2:46 AM

Also, note that printable characters in latin-1 are codes 32 to 126, and 160 to 255. The 127 to 159 are not defined in latin-1. Microsoft defined a superset called Windows-1252 with some extra characters for these values (which then do not correspond to Unicode code points).

@MarkBuffalo Yes.

Byte per byte

Mark Buffalo

Yeah, I previously did it byte per byte, but then I decided to go with Encoding... for unknown reasons

see: laziness when rewriting this after getting frustrated with unprintable characters

Thomas Pornin

@MarkBuffalo Presumably, it could have worked, if you had use a class for a mono-byte encoding (i.e. not UTF-8).

In practice, text data you will encounter will be pure ASCII, UTF-8 or UTF-16, especially in disk sectors.

Mark Buffalo

maybe something like...

public static byte[] byteMe(string str)
{
byte[] b = new byte[str.Length / 2];
for (int i = 0; i < b.Length; i++)
{
b[i] = Convert.ToByte(str.SubString(i * 2, 2), 16);
}
return b;
}

Thomas Pornin

Length, not length

Mark Buffalo

And then I will still be using Encoding.ASCII.GetString(byteMe);

Thomas Pornin

2:51 AM

@MarkBuffalo It is possible that this last call will convert all source bytes which are outside of the ASCII range in some replacement character, which may be a dot, or something else.

It could be a question mark.

Arguably, the decoder could also skip them, or throw an exception.

It is poorly specified.

Mark Buffalo

Yep, I'm getting an exception at the end, even though I'm checking if the last string's length is null or empty. getting a format exception

Thomas Pornin

StringBuilder sb = new StringBuilder();
foreach (byte b in buf) {
    sb.Append((b >= 32 && b <= 126) ? (char)b : '.');
}
return sb.ToString();

What's hard in that ?

Mark Buffalo

StringBuilder was freaking out on me

out of the entire hex collection, it only displays "MZ,?"

I'm probably doing it wrong. Let's see

Thomas Pornin

Probably.

Chances are that the next "character" was a control character that simply stopped the string processing in whatever system you were using for display.

E.g. a byte of value 0.

Mark Buffalo

I think so too

Thomas Pornin

2:56 AM

As a rule, you should try to display only characters that can actually be displayed.

StringBuilder has absolutely no issue with control characters.

Mark Buffalo

@ThomasPornin TIL

Thomas Pornin

The problem is in trying to print non-printable characters. Strangely enough, it does not work.

Mark Buffalo

lol

Thomas Pornin

Time to leave. Have fun with your characters.

Mark Buffalo

Have fun! And thanks!

TildalWave

3:09 AM

if you're into self-flagellation with code you should try epoch conversion between all the time standards next :)

which is another thing that doesn't work but we're doing it all the time LOL

Mark Buffalo

haha

MickLH

You're welcome.

Mark Buffalo

3:48 AM

Had to heavily modify the solution, but it's essentially doing the same in tldr. It works beautifully:

public static string ConvertHexToString(string str)
{
StringBuilder sb = new StringBuilder();
byte[] b = new byte[str.Length / 2];

for (int i = 0; i < b.Length; i++)
{
b[i] = Convert.ToByte(str.Substring(i * 2, 2), 16);
sb.Append((b[i] >= 32 && b[i] <= 126) ? (char)b[i]: '.');
}

return sb.ToString();
}

Thanks again, man!

and now, I have a near fully functional hex editor

!

David Freitag

4:21 AM

Aaaaaaaahhhh god it feels nice to finally have my desk setup

Mark Buffalo

nice

@DavidFreitag @ThomasPornin answered my retarded question, so no need to worry about answering it

David Freitag

@MarkBuffalo You had a question?

Also, there are no stupid questions.

Mark Buffalo

I wrote a hex editor

Took me 4 hours

Well, right now it's a hex viewer

David Freitag

Ok, you want a cookie?

Mark Buffalo

No... thinking of making it MDI

so you can view multiple files at one time

...I have this sick obsession with MDI

David Freitag

4:29 AM

MDI?

Mark Buffalo

Multiple Document Interface. Like how mIRC handles it's windows

oh

I really like it

meh

...so apparently there is a ByteViewer() control in C#...

and I didn't need to write a hex editor

screw that. I'll use my own code anyway

David Freitag

4:36 AM

God my mech keyboard is LOOOOOOOUD AS FUCK

@MarkBuffalo Yeah but chances are it's limited in some incredibly annoying way

@MarkBuffalo you should put your code up on GitHub, and when you finish the hex editor code you can make a Nuget lib

Mark Buffalo

Yeah I suppose

I'll upload it soon, but it's very rudimentary

David Freitag

Mostly I want something to do

Mark Buffalo

Right now it has two separate views

Mark Buffalo

4:53 AM

It's awful

Still some things that need to be done better/right

David Freitag

5:04 AM

Well put the code up so I can give it a looksee

Mark Buffalo

currently slightly refactoring. gimme some timesausage

David Freitag

Bro it's literally three commands

Make the repo, clone it, then git add ., git commit -m "title" -m "body, `git push <url>

I have to admit, as awesome as Windows 10 is, AMM is sooooo much better

Mark Buffalo

AMM?

David Freitag

Actual Multiple Monitors

It's not free but gat dayum it's good

Mark Buffalo

Ah, nice.

David Freitag

5:09 AM

Also, not only is it not free, but you have to pay them for each major release

So I say screw them, I paid for it, I'm downloading the new release anyway.

Mark Buffalo

hm. maybe I should reverse-engineer what they are doing

:D

or make my own for free

David Freitag

It's a pretty big program, likely full of TONS of windows "magic"

Mark Buffalo

loads up git

Alright, David... it's going up

As soon as this loads.

David Freitag

Oh baby ( ͡◉ ͜ ʖ ͡◉)

Mark Buffalo

What git ignore to use?

Windows?

VisualStudio?

David Freitag

5:19 AM

VS

This one, @MarkBuffalo github.com/github/gitignore/blob/master/VisualStudio.gitignore

The best thing about AMM is that windows snap to the edges of windows

love it

Mark Buffalo

@DavidFreitag There you go

David Freitag

@MarkBuffalo linkydoodle?

Mark Buffalo

slash markbuffalo

slash Hextile

Find it?

David Freitag

Yeah this needs some work... lol

But it works

Mark Buffalo

A lot of work, lol

And the splitting is no longer needed... that is dumb anyway

forgot I took out the part that needed that

Do you like the FileNotFoundException?

David Freitag

5:31 AM

I haven't looked at the code yet

Wanna add me to the contributors list?

Mark Buffalo

oh wait, that is actually needed

yeah, sure. we can collaborate

star it, and I will add you

David Freitag

dern sterred

I added a new branch to make things easier

Mark Buffalo

sweet. now I'm gonna sleep

tomorrow the THING begins

David Freitag

@MarkBuffalo Such snark

Mark Buffalo

5:58 AM

@DavidFreitag It's there in case someone deletes the file after selecting it

Also, I think it would be better to load the Converted contents first, and then display the two stringbuilder contents

Would probably be even better to do all of this side-by-side

David Freitag

@MarkBuffalo Windows won't let you delete a file that's open

Mark Buffalo

6:57 AM

@DavidFreitag If you open the file browser dialog, and then delete the file for no reason, then you can do that

because it doesn't get loaded ;b

David Freitag

wut

Mark Buffalo

file browser dialog only opens a list of files available to load

you can select the file, delete it, then attempt to load it. why you would do that... I have no idea

mmm white castle

David Freitag

@MarkBuffalo Then your open function is shite

Mark Buffalo

No, no. The OpenFileDialog does not open the file until you click "ok"

It allows you to select it, but will not lock the file. If you delete the file, and then attempt to load it, you'll have a funny error like that. Low chance of it happening, I just wanted to be snarky.

David Freitag

@MarkBuffalo No, you need to throw in a if (!File.Exists(<filename>) between getting the name from the ofd and trying to open it.

Mark Buffalo

7:15 AM

Yeah, you can do that

I'm kind of zonked out so I didn't make it as good as it could be

Dude. This guy is awesome.

Richie Jackson "Death Skateboards" Part

►

There are even funny bloopers at the end

David Freitag

7:37 AM

> Enter file path: G:\Downloads\as-installer-7.0.582-full.exe
Success, read 761898080 bytes. Took 16590ms. (43.7976269207495MB/s)
First 8 bytes:
4D 5A 90 00 03 00 00 00
Press any key to continue...

LOL, and it's using 3,979 MB of ram

Mark Buffalo

your version, or mine?

David Freitag

Mine

Mark Buffalo

kek

David Freitag

Uses a List<byte> under the hood

which I think is possibly a huge mistake

Mark Buffalo

well I could probably make do with a better version of this

using a stream for example

BUT, on huge files... it can't be helped

zzzzzzzzzzzzz

David Freitag

7:43 AM

Lets try again with an array instead.

wow

> Enter file path: G:\Downloads\as-installer-7.0.582-full.exe
Success, read 726.602630615234 MB. Took 11458ms. (63.4144380009805MB/s)
First 8 bytes:
4D 5A 90 00 03 00 00 00
Press any key to continue...

Only 2,146 MB of ram this time :d

@MarkBuffalo No, the best way to do this is to not load the entire file at once

Only load the bytes you are looking at

However that makes doing things like searching for symbols a PITA

David Freitag

8:36 AM

> Success, read 726.603 MB.
Took 0.363 seconds. (2001.66013943591MB/s)
First 8 bytes:
4D 5A 90 00 03 00 00 00

Now that's what I'm talkin' 'bout @MarkBuffalo :D

Turns out converting the bytes to ASCII takes waaaaay longer than just reading them from disk.

> Converting to ASCII...Done.
Took 9.852 seconds.(73.7517895468163MB/s)
First 8 characters:
M Z . . . . . .

CodesInChaos

9:07 AM

What do you mean by converting to ASCII?

@DavidFreitag Looks like we have a Mark Zbikowski fan ;)

David Freitag

@CodesInChaos Hmm?

CodesInChaos

Printing his initials

David Freitag

Heh

No that is the beginning 8 bytes of a random ~730MB file on my hard drive

Huehuehue, the latest Oglaf is great :]

CodesInChaos

Yet, they're the initials of that dude.

DOS MZ executable

The DOS MZ executable format is the executable file format used for .EXE files in DOS. The file can be identified by the ASCII string "MZ" (hexadecimal: 4D 5A) at the beginning of the file (the "magic number"). "MZ" are the initials of Mark Zbikowski, one of the developers of MS-DOS. The MZ DOS executable file is newer than the COM executable format and differs from it. The DOS executable header contains relocation information, which allows multiple segments to be loaded at arbitrary memory addresses, and it supports executables larger than 64 KiB; however, the format still requires relatively...

David Freitag

> Success, read 396.553 MB.
Took 3.628 seconds. (109.304MB/s)
First 8 bytes:
50 4B 03 04 0A 00 00 00
Converting to ASCII...Done.
Took 5.342 seconds. (74.233MB/s)
First 8 characters:
P K . . . . . .

Real data from a file that wasn't cached in ram

@CodesInChaos LOL, I guess I shouldn't be surprised

CodesInChaos

9:16 AM

zwitching to zip files now

David Freitag

you nurd

CodesInChaos

Initials of Phil Katz

David Freitag

Man what is it with these people

Don't they know that two byte magic numbers aren't good enough?!