I have a json file (40mb's) containing an array of urls, I want to read it and and then write the data in sqlite3. at first I was doing it in nodejs .. I was streaming the data and inserting into database.. but the problem is its soooo sloooww and its like it will take forever to complete .. so I think maybe commiting the data in chunks of transactions (like say I have four process that reads 10mb's of json) and then write it to sqlite in a transaction..
but I am confused as to how do it like .. like one process has read urls[0] to urls[999] how do I make sure that the other reads next the …