I create a view to display ValueRange like this:
...
ValueRange=case when Value1>60 and Value1<=65 then '60-65'
when Value1>65 and Value1<=70 then '65-70'
when Value1>70 and Value1<=75 then '70-75' END
...
But I don't have any data for Value1>65 and Value1<...
@ypercube I must admit I'm curious about the motivation. Apart from the title the question doesn't actually have anything to do with repairing sql server.
@ypercube @MarkStoreySmith, @MartinSmith, @JamesLupolt, @MartinC, @JackDouglas - Yes, the drinkies are still on for next friday. 7:30pm in Richmond at the Pigs Ears
I'm going to assume that you already have the budget to implement some data warehouse solution. Just briefly talking about Vertica; it is a load and read optimized platform, and certainly not designed for OLTP.
The piece on staging and processing data would need some more thought. Vertica isn't...
@Kermit Actually, I think OP is not a large bundle of sticks, and there is actually a question hidden in there. OP seems to want to know if Vertica or a similar platform can be used for the ETL work or whether he needs a different DB platform for the ETL.
The answer would be quite nuanced and come down to 'it depends', but in about 95+% of cases it would be 'no'.
@ConcernedOfTunbridgeWells I'm somewhat confused by this statement: "ETL tools tend not to support complex transformations well." I guess you mean something like that they have a lot of support for complex transformations (at least SSIS seems to), it's just inflexible compared to stored procedures?
@James It's a combination of the low-level primitives available and a phenomenon called the 'Deutsch limit', which describes a notional upper limit to the complexity of computations that can be expressed in visual lanaguages without it becoming too confusing.
Basically, beyond a certain level of complexity you tend to have to use stored procedures for transformations at the ETL tools don't do it well.
@ConcernedOfTunbridgeWells That makes sense intuitively. The larger SSIS packages I've worked with were harder to follow than a really long & horrible query execution plan.
@James It's a fundamental issue with visual lanaguages. Transactional integration tools like BizTalk or Websphere Integrator, and visual rule or workflow building tools have the same class of problem.
@James I haven't tried BIML, but as I understand Ab Initio has its own scripting language, and Qlikview comes with a data transformation engine that uses a scripting language.
Although, XML as a programming language lexicon makes my stomach churn. They might at least have gone to the trouble of writing a parser. it's not that hard.
@Gonsalu Kind of glad to hear it since I don't really want to support that stuff.
@ConcernedOfTunbridgeWells The Qlikview scripting language seemed OK. A funky SQL dialect with lots of nonstandard built-in functions. It didn't even occur to me to use it for any non-trivial transforms, to be honest.
I'm installing it to test a program someone wrote.
@James Oh? I had problems with it. And somewhere I learned that the optimizer is known to have problems. I think Tom Lane was giving a talk on it, or something.
@FaheemMitha I guess they all have problems, since choosing an optimal (or even acceptable) query plan quickly isn't easy. I don't know much about PG specifically, though, just that the PostgreSQL DBA sitting next to me would usually say "you wouldn't have that problem in Postgres" every time I complained about MySQL creating an apparently unneeded temp table or not using an index, etc. : )
@FaheemMitha It's not bad. Backups are great, restores are reliable and easy, and I like the HA technologies. When something misbehaves it's usually for a better reason than MySQL.
@FaheemMitha I don't blame you. I fell into the MS space by convenience (the Linux jobs where I was working at the time were focused on supporting a mediocre virtual hosting control panel).
@FaheemMitha Yep. It's a nice place mostly. Lots to do, aesthetically quite pleasant in the central area. On the other hand, crazy expensive and you have to put up with the generally mediocre quality of engineering in the UK and low UK salaries if you're on the perm job market. The contract market is decent, though.
@FaheemMitha Yep, I'm just a DBA. I'll be renting for as long as I live in London. The prices are very high here: londonpropertywatch.co.uk/avg_prices.html Hopefully saving enough to buy a place whenever I live next.
I've read property in Bombay is fairly expensive as well, though.
Though I know the crowding in India can be much worse. A friend of mine from Delhi says some of the train lines here are worse than some in Delhi, though.
@FaheemMitha DBA salaries around London seem to be about £40k at the lowest end to around £75 at the highest (probably hedge funds and prop trading firms). Bearing in mind the cost of living here and high taxes, that doesn't leave most people with a lot left over. The better money is probably in the contract market. There's lots of work and the rates are around £350-450/day.
@FaheemMitha Yep, it gets busy during the week, esp when UK & US working hours overlap.
@FaheemMitha Not really, you just have to experience in the right industry I guess. The contract market here for software development and IT in general is pretty strong.
@James would you consider contract work? I'm surprisred there is a lot of work though.
@James I see. I think my friend is a contractor for a bank. I think he got good pay, but I guess London is pretty expensive, so maybe he doesn't save much. Plus, taxes.
Is there a lot of telecommuting, or do people want you to come in?
@FaheemMitha I'll probably switch to working as a contractor at some point. I think the main reason there's a lot of contract work is that employers find it favorable: they get generally better people, they don't have to pay benefits or vacation or match payroll taxes, it's easy to let people go with no severance, etc.
@FaheemMitha It seems to vary a lot but for operational jobs I think management usually wants you in the office. There's probably more remote dev work, but in some sectors like banking they don't want anyone off-premises with the code.
@FaheemMitha Yep, the trade off is higher pay, and if it sucks, you can leave without having to explain why you left a job after a few months -- just say the contract expired. : )
I create a view to display ValueRange like this:
...
ValueRange=case when Value1>60 and Value1<=65 then '60-65'
when Value1>65 and Value1<=70 then '65-70'
when Value1>70 and Value1<=75 then '70-75' END
...
But I don't have any data for Value1>65 and Value1<...
I'm going to assume that you already have the budget to implement some data warehouse solution. Just briefly talking about Vertica; it is a load and read optimized platform, and certainly not designed for OLTP.
The piece on staging and processing data would need some more thought. Vertica isn't...
