AI's top startups are barely publishing their research

I can’t speak for other startups, but I applied to the most recent YC batch with my idea for making AI proactive instead of reactive, and pre-being selected I’ve published a paper on recursive self-improvement mapped to the Epoch AI data.

I contacted a professor from a university in the UK and he responded since he was working on similar work, then asked me if I wanted to meet with him. We talked for about an hour since we had overlapping results and different methods, specifically different assumptions.

I say all that to say, as a physics student getting my undergrad, simply doing independent research and speaking to experts about it enabled me to network with someone I otherwise likely wouldn’t know. For young people getting into any business, research is a great way to meet new people.

  From: rms@prep.ai.mit.edu (Richard M. Stallman)
  Date: 25 June 1986 at 18:45:28 GMT+2
  To: a2deh@ai

  I figured out what a copyleft is.
  It's the sort of thing that I put on GNU software and manuals.
  The left-wing version of the copyright that rightists use

  LATENT (L)
  All rights unreserved.
  This invention may already exist, but hasn’t become obvious yet.

  Latentleft — publish the invention, conceal the monopoly.

  Latent Pending — the idea is out there somewhere.

  Latent Troll — remains dormant until an industry becomes profitable.

I've been at two startups that have done genuine world first fundamental research.

The first tried to publish novel results for 3 years in tier 1 journals before finally doing a preprint and telling the tier one publishers to jump in a fire.

The second, and ongoing, isn't publishing anything because of my experience with the first.

That and avoiding openAI and Anthropic copying our results and leaving us with nothing to show for six months of work. The papers only come with the pitch deck.

What the blogificafion of AI research has done is allowed all kinds of claims and terminology related to AI to be introduced and taken up in a manner replicating social media dynamics. And that is simply not healthy. We’re fast reaching a place where any claim can be backed up with a set of numbers from a number of experiments run in some gamified environment or the other, with little concern for if it all adds up to anything. It’s a vicious loop, because this same junk then goes in to train the next models which help spit out the next set of models AND blogs/papers. The net effect is not dissimilar to setting termites loose in a library.

The article is vague about the companies in the paper, for some reason.

In the paper, OpenAI is at the top of the chart for cumulative citations. MEGVII, Hugging Face, Waymo, Momenta, Preferred Netowkrs, Anthropic, Owkin, and Databricks, and Aibee follow (in that order). Yes, that is citations, not publications, but they explain that they're trying to use that as a proxy for significance, albeit an imperfect one.

Companies like Google aren't included because they aren't unicorn startups.

As far as I can tell, the paper never actually mentions the companies who aren't publishing papers. Open AI, Anthropic, and hugging face are all specifically mentioned as companies that do publish papers. Just FYI for anyone else who reads "AI's top startups" and immediately assumes OpenAI and Anthropic.

I’m not sure why this is so surprising? AI company does not automatically mean research company. The vast majority of new startups popping up over the last few years have commercial motivations, and use models built by someone else. Why are you expecting them to publish scientific papers?

50% of startups contributing to public research is actually a crazy good outcome. That’s far more than I had expected.

Yet none of them would have been here if Google hadn't published "Attention is all you need", the irony.

Perhaps I'm imagining it but the entire industry was build on published research, this "AI wave" is at odds with that and seems to be driven by greed (although they'll claim some arms race or something to help themselves sleep at night).

There should be a new ESG (Environmental, Social, and Governance) policy being pushed recognizing the important role this plays. Although ESG and all norms have been set aside in this grim new world it seems.

How much research is a startup expected to do? Isn’t the point of a startup to accelerate productizing research? Unless they are multi billion dollar “startups” do we even expect any research out of them?

There is no reason to publish anything in the AI era until you have reaped as much benefit from it as you are satisfied with. "Building in public" and "Researching in public" now means someone can swoop in with an AI and copy you instantly and become your competition overnight. Keep secrets.

Even at work, I have come to realize if I simply horde my accumulated custom AI built tools and productivity boosting tricks for myself, I can make myself more competitive as an employee.

I think this is how you get hired now, not by having a good resume, but by making claims of having special processes and personal tooling design that gets massive productivity ROI.

Do startups in other fields constantly publish their research?

My worry is learning and innovation. When companies sold physical devices, other companies could learn from them and improve them. We invented patents to provide economic support for innovation.

But, with services, nothing is exposed. Every innovation can be a trade secret. That makes it harder to learn and harder for cross pollination of ideas.

It probably won’t be good for employees either. Another company won’t bid for your skills as high, since you will take a longer time to train to work on their system.

I don't expect non-foundational or non-frontier startups to do, or publish much heavy hitting research. "AI" has become such a ubiquitous description, it is probably easier to find non-AI startups.

There was a good paper documenting this in broader industry, and highlighting how it may account for previous research suggesting good ideas have been getting harder to find. Here's the abstract:

> Relatively flat US output growth versus rising numbers of US researchers is often interpreted as evidence that ideas are getting harder to find. We build a new 45-year panel tracking the universe of US firms’ patenting to investigate the micro underpinnings of this claim, separately examining the relationships between research inputs and ideas (patents) versus ideas and growth. We find that average patents per R&D input are increasing, the elasticity of patents to R&D inputs is flat or rising, and there is no systematic evidence of a secular decline in patenting after controlling for research inputs. We then document a positive, significant, and fairly steady relationship between firms’ growth in ideas (patents) and labor productivity. Average firm growth after controlling for idea growth, however, declines. Together, these results suggest that innovative efforts play a key role in sustaining growth that has not diminished over the last four decades.

https://faculty.tuck.dartmouth.edu/uploads/teresaFort/files/...

And it makes sense. If you publish some state-of-the-art algorithm then you're basically helping your competition, and you don't get anything back from them. So you publish bare minimum in small articles on your website, only to gain people's/investors' interest, and just hope that you didn't share too much. And it'll always be like that, unless the root cause changes - intentives. And the main incentives right now are money and power.

It makes sense when there's at least two posts here on hacker news about model routers which just reimplement Sakana Fugu from its ICLR papers, as a for profit business without any revenue sharing with Sakana nor making them open source. Of course Sakana would want to keep their research private to prevent that from happening. Open router fusion is yet another reimplementation of Sakana Fugu.

All startups barely publish their research, because doing that would be incredibly stupid for the most part, because they want to sell the stuff they invent not give it away for free.

Several things can be true simultaneously:

The process and standards of peer-reviewed publication are tedious. If the objective is efficient communication of knowledge, this is not the process that maximizes that outcome. This process is about credentials and imprimatur, not communication.

Academic publication is largely chasing prestige. The value of prestige accrues more to the individual than the company employing the individual. Incentives for companies reflect this.

Many blogified publications are unqualified slop. This sounds like an indictment but a lot of academic peer-reviewed research is also low-quality slop. In many domains, slop is rewarded, devaluing the contributions of people publishing more serious research, which affects incentives. The signal-to-noise ratio has been poor for decades no matter where you get your knowledge.

Any good research you publish will be exploited against you in myriad ways. It is effectively impossible to assert any IP rights. Research turns into a pure cost center if you treat it this way. Companies recognize this and have adapted to reflect it. In some computer science domains the state-of-the-art has been buried in trade secrets for decades such that the academic literature is embarrassingly obsolete.

Publication costs time and money. Given all of the above, why would you invest productive capacity in public communication of research results? The researchers want to work on interesting problems, the companies want to maximize the leverage from their research. Publishing delivers neither.

A lot of fundamental research is done for the purpose of solving a problem, not publishing a paper. Their motivation and personal reward is solving the problem, not publishing it. Once they've solved it, it is no longer interesting and they are on to the next interesting problem.

Some research is coupled with national security considerations. If you do frontier research this is always sitting in the background.

All of this is a consequence of incentives. A large percentage of all basic research no longer happens in academia. I've greatly enjoyed participating in non-publishing research programs. I also understand why they don't publish. I've done a significant amount of interesting foundational research under my own auspices where it was not worth the effort to publish. I'm not chasing prestige, I had a problem I needed to solve.

No one should be surprised by any of this.

It's not just AI. The whole business of academic publishing is a mess. Can't imagine anyone outside academia wanting to get there. In theory, it should have been a bridge between people who do interesting things that benefit society. In practice, it's a huge indifference machine powered by the institutions KPIs and benefiting no one but Elseveir.

Many AI startups open source part of their work and write blog posts without publishing papers, including the startup I work for. Sometimes, startups just have for more pressing things to do and don't have time to write up scientific papers for publishing.

Also, there are simply too many AI papers that make peer-review publishing quite meaningless these days (e.g., AAAI this year got more than 50K submissions).

While this article is a summary of John Ioannidis’ work, I do find it kind of like Science magazine is complaining that they aren’t making enough of that AI revenue so come on submit your papers here. I doubt that was the intended message.

Outrageous that Science is platforming Ioannidis without a mention of his axe-grinding dismissal of COVID risks and outright attempts at intimidation of opponents. He's already burnt his reputation to slag.

"But University of Alberta AI ethicist Mohamed Abdalla says the findings reflect the incentives facing commercial AI developers, rather than solely a failure to uphold scientific norms. “It’s not the company’s job to advance science, right?” he says. “The company’s job is to advance money.”"

------------

If everyone holds back their publications, the whole sector moves slower. Yes, it's moving alarmingly fast according to many, but is it moving fast enough that the massive investments in data centres will actually pay off?

It's more game theory. Things may go well for one selfish company not publishing in a sea of altruists who publish, but perhaps not if everyone else is selfish too.

Once something goes commercial, academics have to consider what progress would be research-worthy, rather than a half-baked prototype for a product or feature.

Companies can’t be expected to publish their confidential and proprietary information about their feature development, and academics should consider projects that would have higher impact.

If you’re taking up a seat in a PhD program tinkering with would-be feature ideas for an existing major tech company, you should really just get hired by that tech company, where the resources are abundant and the degree is not required.

Considering most of the research is instantly used by competing labs to "update" the way their AI ingest, processes, enriches, or extracts data, we see no surprise here.

It's a jungle out there and no one is following any real ehtical lines on this race to the top. Everyone is trying to consume as much as possible while disclosing as little as possible. Any papers published are probably reviewed 10 times to ensure no "secrets" that could be re-applied are leaked.

There is hardly any new research. Models have been getting worse over time and everyone is stuck on the transformer architecture.

Funny, as ONE PERSON with Claude, I've been on a tear when it comes to publishing my own research:

https://arxiv.org/search/?searchtype=author&query=Magarshak%...

So I know that smart people in those companies can definitely publish. In fact, a whole team should probably be publishing like no tomorrow!

I'm pretty sure there are plenty of folks at Google who wish they never published the transformer paper.

This is also in line with my subjective experience as someone reading quite a few papers in the field: """The preprint also found that firms based in China consistently published more papers than their counterparts based in the United States."""

Startups exist to profit, not necessarily publish.

Honestly part of the challenge here is that they’re not really doing anything truly groundbreaking that everyone else doesn’t figure out on their own weeks later. That’s not an environment that fits the traditional “publish” mindset and is a big problem for these big labs as everything slides towards being an undifferentiated commodity.

We’ve really not had massive advancements in the last few years that weren’t quickly discovered/copied by everyone at the same time, just bigger models. Everyone has been just playing around with the same mathematical parlor tricks from the original breakthrough.

Until that changes I expect we’ll continue to see this arms-race-to-the-bottom as everyone is just battling to make a commodity vs true innovation that gives one startup a legit IP advantage over others.

Trade secrets are back, baby!

More to the point, without determining how much work is “worthy” of a paper it is unclear how much this matters.

Most AI companies are either a product and marketing layer over a model or not meaningfully moving any dimension to be worthy of a paper.

Also, formal papers and blogs and “cards” are all being intertwined.

I think a lot of this “research” is a bit overhyped, when even at Anthropic it’s become “please bro find me publishable research”

eg https://news.ycombinator.com/item?id=49088205

On the other hand, I have a custom harness agent running an ecommerce business 100% autonomously at this point, but have very little to benefit from trying to “publish” anything about it - and without a Stanford or YC stamp, I doubt anyone would care much anyway

I'm under the impression that the majority of startups go nowhere, so the fact that they don't have this property is not itself a major statement about the field as a whole.

Academia is failing to adapt. It used to be that their journals were the only place to learn about technological breakthroughs. Now, I quickly skim the paper and go straight to the source code or data. A publication might be one of the least effective ways to gain scientific impact, second to which is word of mouth.

Also, it's too hard to publish and academics are too hostile. These people are government funded and reject work for things like mentioning Palestine. I've had multiple papers automatically rejected even for pre-publication. Even when I have jumped through all these hoops, it has no more credibility than a blog post to the average person. In fact, most people cannot even read my published work. They look at it for a few seconds, squint, and then say, "Cool."

All of this and the vast majority of publications are not even replicable. I've made do with uploading code or instructions on how to replicate work through git and then make a blog post about it.

I still create papers but they're for myself. When I complete some dense scientific project, a paper is a great format to onboard me onto it a few months later. I'm most concerned about indexing the work, but I find that online archiving tools are the best bet since journals are behind paywalls and are subject to retractions.

Frontier AI research is not only under incredible commercial pressure but it’s also a geopolitical advantage.

I don’t think Dario wants to be a monopolist. But history is full of brilliant people who focused on the social good as priority #1 and lost the game early.

Once you lose the game and go bankrupt you can’t do anything. Same as politics, if you’re not elected you can’t change anything. Of course there are many lines not worth crossing.

YourGPT is one of the AI startups turning research into practical products for real businesses.

the real moat isn't the research, it's the distribution. you can publish everything and still lose, because by the time anyone reads the paper, they're already using your competitor's product.

we are deglobalizing, publishing will become increasingly rare

Who's this'Al' I keep seeing in the headlines?

Why would they waste their time in peer review?

Top startups in China publish a lot?

We are publishing patents though.

Hmmmm I wonder why, so interesting

Do they receive public funds?

> Moonshot AI, one of the Chinese startups included in the study, recently unveiled Kimi K3—one of the strongest open models to date—and publicly released its model weights through Hugging Face today.

...today? (emphasis is mine) And the author was so excited that decided to write up a paper and submit it to Science all in one day? Yeah, right. That's all you need to know who is moonshooting behind this paper.

Why would they? Evil China will steal it... (Sarcasm)

Author requests open weights from Sam Altman

That’s because most of its slop, and not reproducible

[dead]

[flagged]

[flagged]

[flagged]

[flagged]

[dead]

[dead]

[flagged]

[dead]

[flagged]

gate keepers of antiquated practice decry people realizing they they dont need to get past the gate

The age of the dark forest is upon us

> “If we were racing forward on cancer-curing AI, I would be like, ’Fantastic, full steam ahead,’” she says. “But that’s not what we’re racing toward, right?”

We're not racing towards anything. We've been going in circles for years.