It's important to understand why it existed in the first place. A long time ago, if you were building business software on a database at all, it was a very costly commercial database. Imagine 6 figures a year for the software licenses, the same amount again for a support contract, and a similar amount for the hardware to run it on. If a company is spending $1M+ a year on a critical piece of software infrastructure, that is very complicated in addition to being very expensive, they will usually be very open to spending more on top of that in salary to employ experts to look after it. Those experts are your DBAs.
A funny thing happened when MySQL and Postgres made databases "free": companies (irrationally, IMHO) stopped being willing to pay for experts to run their database software. That duty became a secondary responsibility of developers or operators or both, none of whom really wanted to do it or cared to learn how to do it well. There are counterexamples, sure. Some companies get big enough, and lean on their databases hard enough, that they cultivate true competence in them, but it's rare.
And then along comes AWS RDS (and similar), automating away the most tangible aspects of what DBAs did: HA, backup/restore, configuration, provisioning. There's still plenty left for DBAs to do, but it's stuff people don't understand the value of. As above, the residual DBA duties silently fall to developers and operators.
Also, being able to run databases for $0 in licensing on small-ish, commodity servers has led us to use more and more of them, rather than giant, central, shared databases. This amplifies all of the above effects. It also has important 2nd-order effects on the social organization of our software-building endeavors. When databases were shared by many apps, it made sense to think of them as their own services. When each database is built and run to service exactly one app, that makes less sense, plus it's also easier to push everything onto the devs (as above).
Make no mistake, all these companies still need DBAs. Their schema designs are awful, their query performance is worse, who knows if their backups work, etc etc etc. But we don't see it, because there are so few people working in software today that understand what a DBA is, or ever met (and worked with) a good one.
It's rewarding work if you can get it. I still love doing it. But outside of the Oracle and Microsoft ecosystems, it's not really a career.
> When databases were shared by many apps, it made sense to think of them as their own services. When each database is built and run to service exactly one app, that makes less sense, plus it's also easier to push everything onto the devs (as above).
I think this is a really good point. The move towards microservices architectures means smaller and smaller databases. But more importantly, the data is distributed, so a lot of query complexity moves to services and the data is effectively modelled in a nosql style.
I wonder if newsql databases will bring back the database-as-service paradigm. At some point the costs of running hundreds of tiny databases is going to catch up to companies and the prospect of having a single critical instance to care and worry about is going to look attractive. Maybe we will see the return of the DBA in a new avatar over here.
You can probably search around for references on distributed data management. But basically, when your data is split across different instances, you can no longer do joins or transactions or rely on the database for acid guarantees. All these things must instead be handled in code which is exactly how you would work with data using a nosql database as well.
Varies by actual DB but: with this architecture, if the DBs are NoSQL then you didn't lose as much by splitting up the databases, but if they were SQL then among other things you lose transactions across those DBs (or now need costly distributed transactions), effectively making the collective SQL system behave more like a NoSQL DB.
HA is not automated. I know plenty of people who think it is though. None of their shit is highly available. It's easy to handwave and assume it's working automatically though, given all the other hype ignorance. Ask a dev to test the RDS HA. Protip: Take a backup first.
It's important to understand why it existed in the first place. A long time ago, if you were building business software on a database at all, it was a very costly commercial database. Imagine 6 figures a year for the software licenses, the same amount again for a support contract, and a similar amount for the hardware to run it on. If a company is spending $1M+ a year on a critical piece of software infrastructure, that is very complicated in addition to being very expensive, they will usually be very open to spending more on top of that in salary to employ experts to look after it. Those experts are your DBAs.
A funny thing happened when MySQL and Postgres made databases "free": companies (irrationally, IMHO) stopped being willing to pay for experts to run their database software. That duty became a secondary responsibility of developers or operators or both, none of whom really wanted to do it or cared to learn how to do it well. There are counterexamples, sure. Some companies get big enough, and lean on their databases hard enough, that they cultivate true competence in them, but it's rare.
And then along comes AWS RDS (and similar), automating away the most tangible aspects of what DBAs did: HA, backup/restore, configuration, provisioning. There's still plenty left for DBAs to do, but it's stuff people don't understand the value of. As above, the residual DBA duties silently fall to developers and operators.
Also, being able to run databases for $0 in licensing on small-ish, commodity servers has led us to use more and more of them, rather than giant, central, shared databases. This amplifies all of the above effects. It also has important 2nd-order effects on the social organization of our software-building endeavors. When databases were shared by many apps, it made sense to think of them as their own services. When each database is built and run to service exactly one app, that makes less sense, plus it's also easier to push everything onto the devs (as above).
Make no mistake, all these companies still need DBAs. Their schema designs are awful, their query performance is worse, who knows if their backups work, etc etc etc. But we don't see it, because there are so few people working in software today that understand what a DBA is, or ever met (and worked with) a good one.
It's rewarding work if you can get it. I still love doing it. But outside of the Oracle and Microsoft ecosystems, it's not really a career.