Anyscale: The enterprise AI apps are coming

Welcome to Runtime! Today: Why Anyscale brought in a seasoned enterprise tech executive as customers move to the find-out stage of generative AI, a busy week for GitHub, and the latest moves in enterprise tech.

(Was this email forwarded to you? Sign up here to get Runtime each week.)

Network effects

After nearly two years of, let's say, futzing around with generative AI technology, Anyscale co-founder Robert Nishihara thinks mainstream businesses are ready to find out what happens when they start using it in production applications. If those apps are successful, those businesses will have to learn how to operate new types of workloads at scale, and they'll probably need help.

That's one reason why Anyscale brought on veteran networking executive Keerti Melkote to succeed Nishihara as its new CEO last month, adding decades of experience to the AI platform company. Melkote co-founded Aruba Networks in 2002 and in an interview this week he compared Anyscale's position in AI to Aruba's right as mobile computing was poised to change everything.

• Anyscale was co-founded by the creators of Ray, an open-source framework that was designed to help AI developers run Python workloads at internet scale.
• The company, which has raised $259 million according to Crunchbase, sells a managed version of Ray that comes with enterprise features like autoscaling across clusters of AI servers as well as security and governance.
• "There's an infrastructure play that's going to be very big and meaningful in this space," Melkote said. "We have the likes of Nvidia and others that are providing the hardware infrastructure, but what's the software layer that lives on top of that and drives the applications and gives the applications the benefit of the ability to scale?"

Until recently, most of Anyscale's customers were large tech companies that were already using AI in production apps and needed help running those workloads at massive scale. It took some time for companies to learn how to operate effectively across distributed cloud systems a decade ago, and running AI workloads with global reach is even more complex.

• Until ChatGPT arrived, Anyscale seemed like a solution to a problem faced by only a handful of companies.
• "For the first few years of the company, the question people would always ask us is, 'does anyone really need distributed computing?'" said Nishihara, who now works on product strategy. "And for a long time, they didn't."
• Now, of course, generative AI has become a top-down priority inside countless companies, and after a period of experimentation those companies are looking to put their experiments into production.
• "A lot of where we target is this production phase, the people who are who really care about scale, who care about reliability, who have meaningful AI workflows and are getting value out of AI," Nishihara said.

Melkote's job will be shepherding Anyscale's growth as it builds out its sales and marketing operation to target those customers. Aruba had around 1,800 employees when it was acquired by HPE in 2015 for $3 billion, while Anyscale currently has about 120 employees, he said.

• "This company is positioned ideally and perfectly to take advantage of the growth that's about to happen, and it needed that operator perspective of how to scale it up fast," Melkote said.
• The next generation of Anyscale's customers will be far more concerned about cost and ease-of-use than its early adopters, and the product will also need to evolve accordingly, he said.
• And, of course, cloud providers are keenly interested in getting enterprise customers up and running on expensive AI servers, which is both an opportunity and a problem for Anyscale given that Ray is a permissively licensed open-source project.
• Like a generation of open-source enterprise startups before it, Anyscale is likely going to partner and compete with Big Cloud: "the systems and infrastructure challenges are growing harder and harder, and they're going to keep getting harder," Nishihara said.

Stay tuned for more of our interview with Melkote and Nishihara next week.

The August edition of the Runtime Roundtable is live! DevOps walked so platform engineering could run, but building a standardized organizational approach to software development can cause as many problems as it aims to solve if the platform's foundation is shaky. We asked eight experts for advice about how companies should approach platform engineering, and here's what they said.

If you're interested in sponsoring future editions of the Runtime Roundtable, please contact us here.

A launch, a leak, and a letdown

Microsoft's GitHub had an interesting Wednesday, underscoring once again how critical a role it plays at the heart of the world's software factories.

It kicked off the day by launching Autofix Copilot, an AI-powered (of course) tool that scans code as it is written for vulnerabilities, which could prevent a lot of software flaws from reaching production. However, researchers from Palo Alto Networks also disclosed that day that GitHub users may not be aware that they need to take extra precautions to secure authentication tokens that could leak source code or other secrets.

And to top it all off, the service went down for about half an hour Wednesday afternoon on the West Coast after a database configuration change went bonkers. "We are … prioritizing several repair items such as faster rollback functionality and more resilience to dependency failures," GitHub said, adding that "given the severity of this incident, follow-up items are the highest priority work for teams at this time."

Enterprise moves

SAP's Stephan Klevenz is the new governing chair of the Cloud Foundry Foundation, taking over from Catherine McGarvey of VMware.

Shai Morag is the new chief product officer at Tenable, after he joined the company last year when it acquired Ermetic.

Ravi Mayuram is the new CTO of Uniphore, joining the company after similar roles at Luminary Cloud and Couchbase.

Susan Odle is the new chief growth officer at StorMagic, after several years in operational roles at BDO Digital and GFI Software.

Portland's own Chris Polishuk is the new chief revenue officer at Amperity, following similar roles at Lucidworks and Salesforce.

The Runtime roundup

Cisco raised its sales forecast and announced plans to fire more than 6,000 people who apparently weren't working on the things it now thinks are cool.

HPE announced plans to acquire Morpheus Data, a partner that it was already working with closely for its GreenLake infrastructure management software.

Cockroach Labs announced a new licensing strategy that will charge companies making more than $10 million a year in revenue for each processor core running the database, although it will depend on the "honor system" (for now) to collect such payments.

Thanks for reading — see you Tuesday!