Ontology-Driven Observability: Building an End-to-End Knowledge Graph at Netflix

Posted by tosh 3 days ago

Comments

Comment by smartmic 2 days ago

I am in an enterprise which experiments with ontologies and knowledge graphs since its hype heydays ca. 15 years ago. It never took off, good ol’ relational databases and its modern day implementations in the cloud still rule the data landscape. Many smart consultants tried their best and we spend many millions over time into it. I would love to read more success stories, but they seem to be rare beyond academic exercises.

Comment by ezst 2 days ago

Strikingly similar story here. Ontologies and knowledge graphs seemingly make a comeback because it is argued (with little evidence, from what I can tell) that it helps LLMs make sense of the dataset. I see no practical difference or benefit vs. providing the date model as DDL or via other means to represent the ER graph. I'd like very much to be convinced otherwise or at least read a rationale about the alleged differences.

Comment by smartmic 8 hours ago

Ditto. I think this article touches some important points (unfortunately AI-written): https://medium.com/timbr-ai/sql-vs-sparql-in-the-age-of-ai-w...

Comment by ezst 3 hours ago

Thanks for the link. I did go through it despite the annoying style.

It postulates that semantic models delivered in areas where SQL couldn't, that what hinders their broader adoption is the user-interface/querying layer (i.e. too few people adopting SPARQL), and suggests that a new dialect of SQL could solve that, especially because LLMs are so good at producing SQL.

> Knowledge graphs and semantic models promised something enterprises desperately needed: governed meaning, reusable business logic, and explicit relationships across complex data landscapes. And technically, they delivered.

For me that's the main argument: what is the value proposition for knowledge graphs and semantic models when you already have an established relational datamodel? Some argue that the graph's fluid schema brings flexibility benefits, but looking at the problem in its entirety (considering the effort to define and formalise the model, the effort to store content, the effort to retrieve it, deliberately keeping the dev/sysops and tooling out of the way), what you spare at formalisation/insertion stages, you pay back at retrieval (or, if you don't validate or constrain your inputs, expect surprises as an output).

That's where I stand right now:

- convince me that relational models can't express as powerful constructs and concepts as semantic ones

- convince me that "fluidity" in the model don't come with more troubles down the road than benefits

> Knowledge graphs exist. […] And almost nobody queries them directly. Not because the models are wrong. Not because the semantics aren’t useful. But because accessing them requires SPARQL

Or, guess what, a similar outcome can be explained differently: perhaps it's not SPARQL that's too difficult to grasp. Perhaps the alleged benefits of semantic models just do not materialise in practice, and there is no point to engage with SPARQL to begin with because there's no payoff at the end of it. I think the data processing ecosystem is sufficiently vivid and diverse to demonstrate that users have no problem embracing new technologies and languages as the need arise (sed/grep/…, hadoop/spark, pandas/polars/duckdb, decades of ETLs in graphical/textual forms, …)

Then the article is full of weird takes, especially within the "Hidden Cost" section, that I don't feel like commenting. Nor why their "SQL with semantic ontologies" is just syntactic sugar to save your editor from writing joins on your behalf?

Comment by dhenneberger 2 days ago

95% of sparql, cypher, and gremlin can be lowered to sql anyway. I rolled my own sql lowering system for duckdb over iceberg which lets me do everything in that blog post but with no bespoke data layer.

Comment by vladsanchez 2 days ago

What about Palantir!? Care to explain why their Ontology-driven observability is it not considered a Success Story?

Comment by smartmic 2 days ago

Do you have any insights or technical details about their implementation?

Comment by vladsanchez 2 days ago

[dead]

Comment by 2 days ago

Comment by Fordec 2 days ago

I mean, doesn't this exact statement also apply to the adoption of AI and all the winters that have come before this wave?

Also as someone doing my own implementation at the moment, I expect that there is a big relational database backing the MELT store, just the graph aspect is functionally a big RAG interface on top of it acting as a search cache, not itself the data store.

Comment by hodgesrm 1 day ago

[dead]

Comment by 1970-01-01 2 days ago

Without actual examples of it working to reduce latency, this is purely academic and not engineering. What happened to the 1.5s problem in the introduction?

Comment by CharlieDigital 2 days ago

I noticed this with a lot of blog posts from eng teams on AI.

Talk about what they built, not the business problem it solved and how it created value somewhere.

Comment by nout 2 days ago

This seems somewhat outdated approach.

Agreed that centralizing identity is critical (which implies strong reconciliation systems, that can both do well on merges/splits/identity-shifts, but also need to enable backtracing to sources for each triple), but then I don't think it makes sense anymore to force ontology across all systems in the pipeline. As long as you give AI ability to do toolcalls/skills for each individual system and let it build up memory, then AI does much better job piecing together the actual practical ontology. You can give the observability AI to ask each system about it's state for the given piece of information and the AI should be free to make any toolcalls (e.g. call the specific system appropriately via CLI) it needs to gather the precise state.

I think human establishing the ontology gets in a way here, just let the AI poke the systems on its own via clean small interfaces.

Comment by acceptable-scar 2 days ago

[dead]

Comment by j45 2 days ago

I would welcome any and all links or resources about learning more about ontologies, including anyone's personal practice of/with them.

Comment by shervinafshar 2 days ago

SEP entry on "Ontology and Information Systems" is a good start: https://plato.stanford.edu/entries/ontology-is/

If the preference is for books, I can recommend "Semantic web for the working ontologist: effective modeling for linked data, RDFS, and OWL". https://search.worldcat.org/title/1193030282

If the preference is for papers, two papers from Deborah McGuinness et al, from early 2000s are still my favorites:

- "Ontology development 101: A guide to creating your first ontology": https://scholar.google.com/citations?view_op=view_citation&h...

- "Ontologies come of age": https://scholar.google.com/citations?view_op=view_citation&h...

Comment by j45 2 days ago

Thanks so much. I'm open to any information in the space where/how people have seen or experienced it.

Comment by embedding-shape 2 days ago

If you're looking for breadth, here's a bunch of Wikipedia categories that I sometimes revisit to pick out something that sounds interesting and new:

- https://en.wikipedia.org/wiki/Category:Technical_communicati...

- https://en.wikipedia.org/wiki/Category:Knowledge_representat...

- https://en.wikipedia.org/wiki/Category:Information_science

It really is a idea-rabbit hole though, and kind of hard to stop yourself from wanting to create N new projects based on every new idea and perspective you come across.

Comment by dhenneberger 2 days ago

I made OrchidDB to let people express ontology layers over an existing iceberg data lake. It's just a SQL lowering system with a datafusion core for everything else.

Comment by threecheese 2 days ago

Dig into LinkML; I've been eye-deep in defining an ontology for my life (which I struggle to grasp), and just getting into the tools forced me to learn ... outward. There's so much cool stuff, and now I have a mental model as well as a machine-based thing I can talk to (obv an LLM) having a "shared understanding ".

Comment by j45 2 days ago

Appreciate it, added to the list.

Comment by bendriv 2 days ago

It seems to be nearly impossible to find good resources about the subject as they have been crowded out by AI-related SEO-sites. But have you started with W3C’s specs on the subject? Ontologies, linked data, RDF, etc. In Norway we have a government agency publishing local adaption and guidance for these concepts and standards, so that is at least two sources of information on the general concepts and syntax at least. What are you looking for?

Comment by rapnie 2 days ago

Besides AI/LLM the linked data ecosystem struggles with a deep adversion to the technology by the general developer population, dating back from the time when the Semantic Web hype cycle fizzled out. You can see that for instance very clearly on the Fediverse, where the ActivityPub [0] protocol is a W3C linked data based open standard. But ActivityPub can also be plain JSON, and most Fediverse software does not follow real JSON-LD approach. There's a lot of resistance to going the linked data route, being perceived as overly complex. Which is probably a fair assessment too, as the whole set of standards around linked data are large and intricate. As a result of Semantic Web and Linked Data failing to find good adoption, there's a chicken/egg that there's no good tooling to work with the technology and little incentive to create it.

[0] https://www.w3.org/TR/activitypub/

Comment by bendriv 2 days ago

Good point. Here our gov organizations has been required to document their vocabularies, logical information models, data, services, etc and partly also processes, using linked data, so we are working with RDFLib and associated tooling. It has been an interesting ride so far.

Linked Data seems like a good fit for the some of the underlying components in teaching material and technical docs which feed into a knowledge layer for both humans and agents, but also very relevant in our archival/library-like processes. Due to the gov requirements, API surfaces will likely also be documented with Linked Data.

Comment by j45 2 days ago

I found the same - it's definitely studied in academia, some people do seem to practice it, and there do seem to be some standards or frameworks in some places.

Comment by smithclay 2 days ago

It's at the end of this Netflix post: but the end game is to connect whatever your observability ontology is with whatever system the business people and data engineering teams use to be able to answer previously hard questions like "how much money did this operational or engineering change make or lose?"

In theory, at least.

Observability vendors have been beating the drum for 10+ years that there's so much untapped business value in operational data. Seems like we might be getting close to seeing how real that is.

Comment by threecheese 2 days ago

The problem here is we're going to let the machines dictate the ontology, and we will lose expressiveness by reverting to the mean. All in order to communicate more cheaply with the machines - but on their terms.

Forgive the anthropomorphizing, it was necessary - "terms" has double meaning there, and each of us can understand that without an ontology - because we can access nuance. I can read schema, structure, and shape, and understand; but only one of those triggered some readers.

Anyway, the machines will choose terms which are most well-represented in the training data. This will normalize communication by chopping off the top-end where nuance lies. This is token-efficient.

Ontologies, in their implementation in the Linked Data ecosystem, do offer the ability to relate one understanding-system to another, but this is not token-efficient. I would love to see each domain, each business even, define their view of the world in the most expressive and culture matching owl representation, but I don't see it happening. "Shape" it is.

Comment by soco 2 days ago

I have a naive question here: how different are all those business? Does every one need their owl representation? How was it when SAP was modeling every process in the enterprise, wasn't it not already uniformizing enough? Or maybe you mean the value gets lost exactly in those fine deviations from the cookie cutter approximations?

Comment by hodgesrm 1 day ago

I'm curious why the ontology-driven approach is better than just loading records like log and trace data into some variation of SQL tables with full text indexes ad/or Bloom filter indexes. That's how most log management systems work. It's pretty effective, especially for very large datasets.

Comment by vindex10 2 days ago

Just listened to the Pragmatic Engineer podcast on Uber AI infra, https://newsletter.pragmaticengineer.com/p/ubers-ai-infra-fr...

it seems they are using a similar ontology called "context graph", used as a navigation layer for the ai agent.

Comment by 2 days ago

Comment by nico 2 days ago

Is this like XML and UML all over again?

Comment by dominotw 2 days ago

Netflix must have the most overengineered tech stack out there. Ppl hired with such high cash comp must feel the obligation to live upto that comp by creating these sort monsters.

relevant hn

https://news.ycombinator.com/item?id=49878857 - A Staff Engineer's Guide to Inventing Work

Tech industry is overstaffed to the gills . No one really knows how to create value for customers( only for themselves through these overegineered slop for career advancement) . Tech jobs are really sad place to be now.

Comment by acceptable-scar 2 days ago

[dead]

Comment by antonvs 2 days ago

> This agreement has a name: an ontology. Ontology is a formal specification of types, the properties they carry, and the relationships they may have. Its unit is the triple, one fact expressed as subject, predicate, object.

Er... nope.

In the limited context of OWL and RDF, that's what an ontology is. It has a much larger meaning outside of that context. As M-W puts it, ontology is "the branch of metaphysics concerned with the nature and relations of being."

Projected into computer science, an ontology is a formal model of concepts in a domain, their properties and relationships, along with axioms or constraints that apply to those concepts.

The "triple" described in the quote is an implementation detail, how a particular formalism (OWL and RDF) chooses to represent a rather pale shadow of what an ontology is conceptually.

Comment by 2 days ago

Comment by llmslave 2 days ago

Are knowledge graphs still relevant? Agents can grep and fetch on a unix operating system with higher accuracy

Comment by Fordec 2 days ago

My local knowledge graph dropped my token usage spend on reads from being 30% of total spend to 2% and I haven't suffered a regex false positive deleting 300 lines of code randomly since. As of this weekend it also now feeds my QA metrics, replacing what was multiple tool calls taking multiple seconds down to about 25ms because it's a single cached graph traversal now rather than multiple tools traversing the same ground repeatedly.