Before making any decisions about what approach that you would like to take, I always feel it is important for the development team to always consider the option of a complete rewrite as a possibility, even though many think it is nearly always a bad choice.
Take some time to write up some high ...
We have a legacy system to bring up-to-date because:
It uses an unpopular (amoung our users) non-sql database (Btrieve)
Provides only a text interface
Is written in Turbo Pascal (but compiled in free pascal) which is hard to recruit/retain developers to maintain it
To give you an idea of the ...
Converting a non-SQL database to a SQL one is usually an extremely stupid idea. Move to another nosql DB, if you like, but stay away from SQL! – SK-logic 1 hour ago
@SK-logic Many would argue the opposite... – maple_shaft♦ 1 hour ago
@SK-logic Specifically the non-sql (not no-sql) is Btrieve, I don't know if it's more replaceable with a no-sql db, that an sql one but don't get the wrong idea, it's not a "no-sql" db as it stands, it's a record based ISAM db. – weston 51 mins ago Who is paying for the rewrite? i.e. how much budget do you have? – JBRWilkinson 46 mins ago edit
Now your boss comes in and asks you to give him a breakdown of all visitors in a suspended status that visited facility 2 in April 2011, and I want to see a comparison for the previous 4 Aprils.
>We even attempted replacing the low-level DB code with code to convert to SQL, and while it worked fine, it was too slow because the code is still accessing records one at a time generating far too many SQL statements than it would need.
and therein lies another problem, not understanding the way SQL works
There might also be data they store that lends itself to SQL, and other data that may be better off stored in a NoSQL store, nothing wrong with a hybrid solution either.
damn...we're all too rational, might as well put away the popcorn :)
@maple_shaft I worked on a project recently where we used Mongo to store some "difficult" blobs of data that were hard to model in SQL, worked out just fine.
@maple_shaft - I have another project where we have a bunch of health and safety forms (around 60 of them - all different and new ones being thought up all the time) where it's just not practical (or cost effective for the client) for us to build a fancy form builder and all the DB paraphernalia to support this. so we knock out a quick html page then just json'ise the form data and throw it into Mongo. there's some common metadata/fields that we store in SQL, but the form fields are no-sql'd
it's a workable compromise and means the client can afford to have the work done and we get paid
@Kev We had such a project that sounds very similar....
unfortunately it fell through with the client for whatever reason, but their situation would have been similar... thousands of different types of training documents for different schools and hospitals, we needed to give them the ability to build custom forms with dynamic fields
we then needed to categorize these documents into common types and perform business logic based on which documents exist for which students
I sometimes wonder what planet some of these NoSQL "advocates" come from. No everything is "webscale" and most SME biz data still lends itself to Relational SQL just fine. also show me admin and reporting tooling that is as good as what I have with MS SQL [puts on fireproof coat/heads for door]
@MichaelK I have said this once and I will say it again... if you have so much user demand that you need to scale to infinity, then this is a Good Problem to Have ™
Bad Problem to Have ™ - Product is a complete failure, users are not coming, VC is drying up, boss is losing his mind and SCREAMING at every body because they are printing too many color documents... sits alone in his office staring blankly at the wall in the dark.
@MichaelK exactly, was gonna tack on that in the real world most folks are building apps that are supporting real world revenue generating businesses - they're not in that ycombinator reality distortion zone building "omg must scale for anything" fantasy nonsense.
@Kev Yep. In my opinion scalability follows maintainability. You should worry that your code is high quality first. A loosely coupled architecture will naturally scale, because you can easily swap in new components as you identify where the stresses are.
NoSql has its place in the world.. The issue is not more Saleable vs Less Saleable. You need to look at the complete set of requirements. Take Google Search for example NoSQL works for them because of their business requirements.
Take Google Search for example NoSQL works for them because of their business requirements. Their business rules state that 2 people running the exact same query are allowed to get different results. Their business rules state that when indexing updates are made across servers, It’s OK for those servers to be out of sync for days at a time..
You try telling you accounting dept. to follow that same play book in the name of performance... Good luck.
I bet those Linux instances are running on big old IBM z's
I imagine Jeff Bezos leaves nothing to chance there, despite the cleverness of EC2 and the like
@MichaelK back in 1998, the company I worked for (TeleCity) was a tenant in the Bank of Scotland data centre in Edinburgh. We had a fenced off area in one of their modules. Their staff asked us to help move an IBM mainframe (or part of) across the DC floor, apparently this thing was worth about a £1.5 million back in '98. Most expensive thing I've ever touched/pushed/sweated on :)
We have a legacy system to bring up-to-date because:
It uses an unpopular (amoung our users) non-sql database (Btrieve)
Provides only a text interface
Is written in Turbo Pascal (but compiled in free pascal) which is hard to recruit/retain developers to maintain it
To give you an idea of the ...
Amazon's internal infrastructure, from highscalability.com:
Linux
Oracle
C++
Perl
Mason
Java
Jboss
Servlets
SQL vs. NoSQL Flame-o-rama 2012
SQL vs. NoSQL Flame-o-rama 2012