Why are you adding a pepper?

@schroeder Referencing this answer: security.stackexchange.com/questions/3272/… . That way you couldn't upload some random PII to see if it is a match (because you wouldn't have the pepper you need connecting).

I'm thinking you should be using a UUID instead of all that ...

@schroeder Like a username for each user? Application doesn't require a new account for every batch of new PII uploaded.

As a complete aside: you don't happen to be subject to GDPR or something similar, do you?

@schroeder No GDPR compliance. I'd just like to get the tracking user functionality without actually storing anything about them.

I'm not really sure what you are asking. The question you link shows a method for adding a pepper. It sounds like you are trying to get your entire process reviewed.

do you know what a UUID is?

do you need to track them from session to session?

or just within the session?

Yes, tracking from session to session is the intent

ok

Like they upload their PII on day 1 -> create a hash. If they come back day 10 with the same PII -> I can see theyve been here before

ok, as a GDPR expert (I know you are not subject to it) this is a really bad idea in terms of modern data protection, but we can put that aside

I was attempting to create something that acted like a unique user ID.

Interesting... Would you be able to link me to any resources that could figure out a better solution?

your stated question is how to add a pepper - but the linked question shows that - you asked how to do that in Python, which is a programming question and not a security question - you then ask how to "improve the algorithm" but you do not state what you want to be improved - so I need to ask: what are you really asking?

are you asking if this is "ok" to create a unique ID?

if this is all internal, then I'm still not sure why you are adding a pepper - if someone has access to your system, then they can know the pepper and "upload some random PII to see if it is a match".

Yes I guess I was originally thinking about just asking if the pepper was being added correctly. I guess the question kind of spiraled into "is this a good method." So I guess my new question is... is this a good method of creating a unique ID

So in terms of what I am really asking, it seems like what I should be asking is: is this a good method

ok, evolving questions happen

but now your question is not really a security question - I offer 2 counters to your approach: what if someone changes their name (gets married)? or moves?

I applaud your desire to not store personal info

Sorry about that. Yeah that's a good point. I guess I like the idea of adding another level of complication through the Pepper. But seemingly it doesn't really add any new level of security

I personally think that you are trying to solve the wrong problem the wrong way with great intentions

Re: getting married / name change -- definitely valid, just hoping to get something that covers a lot of people short term. And if it's to 100% successful that is ok

sure, ok

*and if it's not 100% succesful

hmmm

I guess creating unique IDs based off multiple, potentially changing, pieces of information is simply -- difficult

Which is why so many US companies use SSN

[[ stifles internal scream ]]

Haha

That's next level terrifying

ok, putting data protection hat on to find a solution: why are you collecting the PII?

broadly - you don't have to get into specifics if you don't want to

I think i'm going to get pretty good coverage just with my hash, keeping it all internal. Even if someone uploads fake data, it won't let them know that there have been any other matching hashes

It's an essential part of the application. Currently I just work with it in memory, but I want to start trying to figure out if people are coming back.

By fake data I mean, uploading another persons name and address to see if they can get a match. Yeah it will mess up my internal tracking system, but it wont reveal anything.

... how will a user "get a match" if you are not storing PII?

match to what?

Another hash that was previously generated

ok, but that;s hidden from the user, yes?

Yes.

So no hashes, or previous data is revealed to the users (just internal).

kk - then, yes, I would just use the hash and not bother about the pepper

OK. What would be the reason to not bother with the pepper? Just plain unnecessary?

note that if DP regulations change for you, then that hash will likely have to be treated as a form of PII

Is only true way to avoid PII while still tracking individuals is to have some sort of non-PII that they create (like a username)?

pepper adds no value to your scenario - unless it was super secret, even from the system - which is what the linked answer explains.

nope - usernames, serial numbers, IPs, etc. - if you can tie them to a person, according to GDPR and many other regs around the world, it's the same as PII

and because you derived the hash from PII, you cannot separate them

UUID would create a potential to psuedonomise if you could break the mapping of the UUID to the personal data

so, like a table linking PII to the UUID, then deleting the linking row, when required

but by creating an ID from the PII, it will always and forever be PII

sucks, huh?

OK. I think the first thing I'm realizing is the definition of PII is more broad then I thought it was

Yeah really. I wonder what other solutions are out there. I guess most companies just go ahead and store the PII

depends on the regs you are under - but the US is shifting towards a GDPR-like approach, as is the rest of the world

tokenization is the solution

and a UUID with a mapping table that you can control is the poor man's version of it

What's the advantage of the UUID with a mapping table? At some point you can delete the reference?

yep

and you can pass on the data to another processor without passing the PII, too

At the end of the day Hash of PII, still pretty much is PII I take it.

yep

With that conclusion I think I need to go back to the drawing board

Thank you for all your help today

Both from a security and a practical perspective around this

because it is not just the leaking of the PII, but the fact that I can go to your system, input my valid info, and see the data you have collected and processed about me

with a token design pattern, the connection to me can be broken

you are welcome

I'm thinking of starting a data protection stackexchange - all this is simply not obvious or straightforward

I think what I can try and do is a combination. First, not let you see if there is any data already on your file. Second, use a pepper disconnected from the app, stored in a secret variable, so that if you did somehow have the database and try to run your info through the hash, you wouldnt know the pepper and couldnt find a match

that could work

what do you want to do with your posted question?

Third, create a UUID v. Hash table, so that after 6 months, the inital link is broken, and only the UUID remains

I will delete it probably. I dont think it accomplishes the goal it set out to ask / is really askign the right question it was supposed to be asking

from a high level - it sounds like an approach that could work

I would like to read a blog that talks about all this stuff more indepth

because trying to piece together the info on the web is ... difficult. I wish I had realized earlier what you had told me, essentially that what I was trying to do wasnt accomplishing the goal that it was

that hashing pii == still PII

I care about doing a good job on this

Cant tell you how much I appreciate your help.

So thank you again.

no problem