This is interesting, makes me wonder if a "dockerised" database is something people could use. I mean a database frontend with its own language/protocols/whatever that allows you to define the data structure but leaves the specific storage engine or format as a backend detail that can change from platform to platform.

> I mean a database frontend with its own language/protocols/whatever that allows you to define the data structure but leaves the specific storage engine or format as a backend detail that can change from platform to platform.

That's more or less a description of SQL.

Postgres wire format is indirectly getting there. Plenty of tools use that with wildly different storage engines on the other end.

A clean room implementation would likely yield different results but there appears to be some appetite for a solution.

Nah. It's not that. We lack a concept that can organize storage. Let me illustrate this.

So, until some years ago there was complete nonsense and anarchy in Linux networking management. That is until we got the "ip" program. There's still nonsense and anarchy, because the "ip" program doesn't cover everything, but it's on the right track to organize everything Linux knows about networking under one roof. So, vendors today, like, say, Melanox (i.e. NVidia) choose to interface with "ip" and work with that stack rather than invent their own interfaces.

When it's extendable in predictable and convenient ways, user will extend and enrich functionality.

Now, compare this to Linux storage... I want to scream and kill somebody every time I have to deal with any aspect of it because of how poorly mismanaged it is. There's no uniformity, plenty of standards where at most one is necessary, duplication upon duplication, layers... well, forget layers. Like, say, you wanted a RAID0, well, you have MD RAIDs, you have LVM RAIDs, you have ZFS RAIDs, you have multipassing with DM (is that a RAID, well sorta' depends on what you expected...) also, well, Ceph RDB are also kind of like RAIDs, DRBD can also sort of be like a RAID...

Do you maybe also want snapshots? How about encryption? -- Every solution will end up so particularly tailored to the needs of your organization that even an experienced admin in this very area your org is specializing will have no clue what's going on with your storage.

Needs can be studied, understood, catalogued, rolled into some sort of a hierarchy or some other structure amenable to management. We haven't solved this problem. But we have an even bigger one: no coordination and no desire to coordinate even within Linux core components, forget third-party vendors.