The Heap™ – Consultancy ©®

Zikato

05:57

Morning

Paul White

Morning

@ErikDarling No, just coincidence

Erik Darling

Weird how databases seem to attract so many people who insist on doing things the wrong way

In exactly the same way

Paul White

06:20

I saw a comment from Martin Smith saying the TOP (1) WITH TIES thing is popular on Stack Overflow

This is what happens when you let programmers near a database

Erik Darling

Defensive programmers <> Programming defensively

2

That row number sorting thing is used in EF

Or was at some point. I recall dealing with it.

Paul White

@ErikDarling Specifically, the ORDER BY ROW_NUMBER syntax?

EF probably copied it from SO as well.

The first time I remember seeing it, it was posted by M. Smith as an aside.

I bet he regrets doing that now.

Zikato

Top with ties

twitter.com/Zikato25/status/1438790662271295493

2

Paul White

ha ha ha

I had already RT'd that but forgot about it

Zikato

06:53

It was one of my well-received jokes. Those are rare

ypercubeᵀᴹ

07:15

rare and well done ;)

Zikato

08:27

heh
https://stackoverflow.com/users/5936629/zikoat

Erik Darling

@PaulWhite yea. Young Josh can confirm.

Paul White

Someone should find the ringleaders responsible for EF and bring them to justice

Erik Darling

I don’t think rounding up a posse is an unreasonable reaction

Paul White

Being reasonable is overrated anyway

Erik Darling

The League Of Unreasonable Gentlemen ride again.

Zikato

09:08

The DATEDIFF saga continues

twitter.com/dhmacher/status/1579385082829484034

Paul White

You'd think people had never used SQL Server or looked at an execution plan before

These people claim to be fucking performance consultants

I find it all very depressing

Erik Darling

10:23

Yeesh.

Paul White

I suppose it takes people time to get up to speed on data types introduced in SQL Server 2008

Josh Darnell

13:09

@ErikDarling Yeah, that happens in EF6 when you use the Take() / Skip() LINQ methods (AKA FETCH / OFFSET). In EF Core, the generated query doesn't have that weirdness, which I had not noticed until now cc: Paul White

Erik Darling

Someone really mastered paging queries eh

Sean Gallardy

13:33

Interns these days...

Sean Gallardy

14:12

As you all know, I'm "learning" Git. I accidentally added a file git add file.foo for me, the opposite of add would be remove. I didn't rtfm, tried remove... doesn't exist. Here's what bothers me, add... well... adds. Remove is... restore --staged wtf?

John K. N.

14:32

rm file.foo

rm linux short for remove

Sean Gallardy

I'm speaking about git :)

Cade Roux

remove meaning to remove a file from staging

I still mostly use github desktop and miss TortoiseHg, which for some reason was so much better than TortoiseGit, even though there is basically a 1-1 correspondence from Mercurial to Git.

Sean Gallardy

It's just odd that it's so asymmetric. Add a file to stage is "add" but remove a file from stage is "restore --staged"

I would understand "remove --staged"

Cade Roux

I think effectively in github desktop we don't do add at all because it is atomic with the commit - whatever is checked off is added and then the commit is performed, and so there is never any remove at all.

Sean Gallardy

@CadeRoux This one? desktop.github.com

Cade Roux

14:44

Yeah

Sean Gallardy

Awesome, downloading. Does it handle rebase merges well?

Cade Roux

Good question, I don't ever rebase

Zikato

@SeanGallardy I think it's rm --cached
But yeah, it's confusing. Once I stray off the golden git path, I usually google

Cade Roux

The medical informatics guy on my team's repo was so messed up I just had to blow it away and clone fresh. I couldn't get it to pull and update - He only directly commits wikis for the documentation to it and so it's always way out of date.

I can't imagine having to teach him how to use the command-line.

Yet he has used Perforce for many years

Sean Gallardy

Yeah I decided I should learn the commands first via command line, since GUIs are great but also obfuscate what is truly happening... though I guess ignorance can be bliss (and efficiency?)

Cade Roux

15:08

I guess git has a lot of complexity, but really, all I need is the basics. Branch, commit, merge, diff. That's what I miss the most about TortoiseHg - the graphical branching/merging view.

ypercubeᵀᴹ

the opposite of git add <filename> is git reset <filename> I think. But yeah, git rm --cached <filename> works, too.

it's not true undo add though. There are several helpful answers here: stackoverflow.com/questions/348170/…

The answer by leonbloy explains why the action of git add is not always undoable.

Sean Gallardy

15:25

wow.

Erik Darling

16:51

Nerds

Sean Gallardy

2

J.D.

The Git when EF 6 introduced that row_number sorting thing Erik: github.com/dotnet/ef6/commit/…

If that's what you were referring to?

Erik Darling

17:07

@SeanGallardy a friend of mine’s dad was an antique book dealer and one of this best clients was booger

@J.D. thanks I hate it

J.D.

Np, me too, so I wrote a verbose (not unusual for me) reply about why it's wrong in the comments lol.

But I'm probably a few years too late.

Erik Darling

Well thanks for defending SQL Server’s honor

Josh Darnell

At least they didn't re-make the mistake in EF Core.

Erik Darling

Even though it appears she’s already been mounted and rode by this one

@JoshDarnell does anyone use that?

Josh Darnell

17:23

@ErikDarling The company I work for is using it in several projects.

J.D.

Yea we're using it too, but slowly migrating to encapsulating more database layer stuff in stored procedures to avoid some of the generated SQL and have better control on tuning.

Erik Darling

How do you find it comparatively? I’d love to be able to tell people that it’s a better option, but have no basis.

Josh Darnell

I think a lot of places are using it for new development, but I haven't heard about a lot of migrating existing apps from EF6 to EF Core.

@ErikDarling I mean, it still has a lot of the same problems. There are some better defaults in place, which helps people avoid getting shot in the foot. But I have seen people I work with just change the default straight back to what they were used to in EF6.

J.D.

I've been using LINQ2SQL and EF for most of my career. I think ORMs certainly have a place in general. I haven't found anything specifically different (better or worse) with EF Core vs previous versions but I've been doing less application development lately too.

I think right tool for the right job though too. It makes development easier for devs, but isn't always the solution, and can certainly be abused with horridly written code lol

user

Does going to NoSQL improve developer performance? I mean, does it make development faster/easier?

J.D.

17:34

biting my tongue hard - It depends.

Josh Darnell

A good example is that automatic lazy loading is turned off by default in EF Core - you have to explicitly include related entities when you write the query, which cuts down on the N+1 query problem where tons of queries get fired off at SQL Server. But I think people just tend to...turn lazy loading on instead of being thoughtful about each query.

J.D.

lol fair.^

Josh Darnell

Another one - in EF6, if you write a WHERE clause that couldn't be translated into SQL, it will automatically fall back to pulling the whole table into memory in the app and filtering there. In EF Core it throws and error telling you it won't translate that query. But it's pretty easy for folks to manually pull it into memory (add .ToList() to the query) to "solve" that problem.

So it's the same problems, although maybe it will motivate some folks to get educated about what the defaults are trying to point you towards.

The bad thing is that when someone googles for that lazy loading problem in EF Core, they're going to get a highly upvoted Stack Overflow answer telling them to do the crappy solution 😀

@J.D. Absolutely.

user

I m asking about dev-speed in NoSQL because of this paper by MongoDB. They claim:

1. The RDBMS optimizes data storage efficiency (as it
was conceived at a time when storage was the most
expensive component of the system.

2. MongoDB’s document model is optimized for how the
application accesses data (as performance, developer
time, and speed to market are now more important than
storage volumes).

To what extend is the "developer time" and "speed to market" true? Or in which cases? We are thinking about migrating our PostgreSQL to Mongo

Josh Darnell

I haven't worked with NoSQL enough to really know how true that is. I built some toy stuff with RavenDB a while back. On the one hand, it was nice to not have to worry about DB schema. On the other hand, I don't have any idea how it works as the data model evolves.

Erik Darling

17:45

@user why would they lie to you.

@JoshDarnell @J.D. thanks. So it sounds like new boss is same as old boss.

user

@ErikDarling Exactly. That's what I told my colleagues today and they were laughing :P

They have a million reasons ($) to be biased, but I guess it should be close to the truth. Right?

Erik Darling

It sounds like you should be using MemeDB which requires no developer time at all

4

J.D.

@JoshDarnell Yea haha, unfortunately so true, most people will go with the ToList() solution regardless if they understand the implications or not, just cause it's an easy fix.

"fix"

@ErikDarling 👍

@user I'd say ask your colleagues "what problem are they trying to solve?". Is development time the biggest bottleneck your team / organization encounters, and has it been proven that is primarily due to managing the structure of your data both in the application layer and in the database layer, and a solution like an ORM wouldn't help solve the problem for x, y, z reasons?

mustaccio

18:09

> Does going to NoSQL improve developer performance?

with eventual consitency, yes

You can trawl this site for the questions tagged with whatever "NoSQL" beast you are interested in, for the questions of the kind "but how do I do X now?" -- it'll give you an idea of productivity

In short, developer performance is determined by the factors other than the choice of the database (provided they know their database of choice and not just listed it on their respective résumés.

Also, how do you even measure developer performance? I sure hope it's not by the LOC per day...

user

I can't measure it. I think we are doing pretty well. But I m wondering if it can be better as the paper claimed. I m not experienced at all, but I wonder if we would be spending less time by having fewer tables and increasing a bit data duplication (due to the not so complex relationships of our data and the not so big data). It's very like i m completely wrong.

Erik Darling

Have you tried being right?

J.D.

lol

@mustaccio 🤙

mustaccio

> I can't measure it. I think we are doing pretty well.

That doesn't compute

If you don't measure it, how do you know how you're doing?

J.D.

18:24

@user Nothing's free without tradeoffs. Increased data duplication means increased developer time spent on ensuring data accuracy in the consuming applications and reports. Whereas normalized data in an RDBMS may have prevented that extra time spent. Just one potential scenario.

user

I can't have an objective measure of how well we are doing. I don't think it's quantifiable. What i do have is client satisfaction which is a simple indication we are doing ok. So, no hard numbers. Is there a way to have a better metric?

mustaccio

So, how do you measure client satisfaction then?

user

Features get implemented as requested without (many) complaints.

mustaccio

So, you expect that replacing the database layer with a new one will let you implement more features with fewer complaints?

Go for it then

Don't forget to track the team velocity and the number of complaints, so you get an objective measure of the NoSQL developer performance.

user

We are going to microservices, so we are rebuilding some parts. We'd prefer to use the best tool for the job.

mustaccio

18:31

The best tool is the one you know how to use

Erik Darling

What happens when someone needs to report across all those micro services

Seen that too many times

Plug it all back into a relational model

ypercubeᵀᴹ

@user big mistake

user

@ypercubeᵀᴹ what are your thoughts on "A MongoDB White Paper - RDBMS to MongoDB Migration Guide?

ypercubeᵀᴹ

18:47

@user I'm sure it'll be a joy to read

Erik Darling

That’s a 20 page pamphlet that ends with a dozen ways to give them more money

J.D.

@ypercubeᵀᴹ Which one is the big mistake, NoSQL or microservices? (Serious question lol, I'm not super familiar with the pros / cons of microservices.)

Erik Darling

I am currently preparing an invoice for pain and suffering

Which is difficult to put a price on

Probably 10-20 million but I’m willing to settle out of court

user

what about this short passage? "The RDBMS optimizes data storage efficiency (as it
was conceived at a time when storage was the most
expensive component of the system. MongoDB’s document model is optimized for how the
application accesses data (as performance, developer
time, and speed to market are now more important than
storage volumes)."

Btw, i m not insisting because I think any of you are wrong. Reason i m asking is because I'm trying to rule it out as an option based on specific arguments (which I am gathering from everyone that's been answering my questions so far)

J.D.

19:10

I think just loosely spewed generalizations that are insignificant when actually compared under a microscope. "Storage efficiency" in this context probably alludes to the fact that RDBMS prefer a normalized data structure where data isn't duplicated. One can argue storing all the related data in the same place (e.g. table), even if duplicated, is more performant to read from, but that's not a unique implementation in NoSQL, you can design your tables just the same in an RDBMS.

Erik Darling

Storage is still the most expensive component in most systems

RAM is cheap. Buy more.

ypercubeᵀᴹ

"The RDBMS optimizes data storage efficiency" is completely wrong in my opinion.

Other excerpts from that guide that reveal the author's lackness of understanding what the relational model is:

2

> Modeling this real-world variance in the rigid,
two-dimensional schema of a relational database is
complex and convoluted.

Assumptions (that the database is accessed only by one application):

> MongoDB allows schemas to evolve dynamically, such
operations requires upgrading just the application, with
typically no action required for MongoDB.

J.D.

19:28

@ErikDarling Enterprise licensing 😉

Paul White

20:13

One good goal is to slow developers down to the point where they're almost compelled to do some thinking

5

Sean Gallardy

Has anyone been successful in setting up a TPC-H benchmark/workload test?

Hannah Vernon

@J.D. regarding performant

Josh Darnell

20:29

@PaulWhite Hahaha that's a good goal.

J.D.

@HannahVernon haha fair enough. Aaron's comment enlightened me too: "Why do you need to find the inverse of the non-word "performant"?". I personally like how it sounds though and find it as real of a word as irregardless. 😁

Paul White

20:57

> Irregardless

Wow

J.D.

😉

Erik Darling

21:43

@SeanGallardy yes

Sean Gallardy

@ErikDarling I'll have to get with you then, I want to setup some TPC-H benchmarks for some testing I want to blog about.

Erik Darling

22:39

It should be easy with hammerdb or benchmark factory

Sean Gallardy

23:25

@ErikDarling ah, I thought maybe you'd done this before: tpc.org/tpc_documents_current_versions/download_programs/…

Transcript for

The Heap™ – Consultancy ©®