New York kicks off the AI audit era

Hello and welcome to Runtime! Today: Companies that use AI software to hire New Yorkers have new rules to follow, China's big cloud players roll out generative AI tools, and the quote of the week.

(Was this email forwarded to you? Sign up here to get Runtime each week.)

Concrete jungle where forms are filled out

New York is about to conduct an interesting experiment in algorithmic transparency. A new law that went into effect Wednesday requires businesses using AI tools to make hiring decisions to show their work.

The law forces companies hiring New York residents to conduct annual audits of any software they use to automate steps in the hiring process, such as automated resume screeners or chatbot interviewers. The goal is to bring more transparency to the process used by those tools to make recommendations, according to The Wall Street Journal, given that such tools have a well-documented history of exacerbating human biases.

The law specifically targets "automated employment decision tools."
That means tools that use “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons," according to the National Law Review.
Any company that uses that kind of software in the hiring process must publish the results of "bias audits" that evaluate how the software complies with guidelines set out by the U.S. Equal Employment Opportunity Commission.
They'll also have to inform prospective applicants somewhere on their web site that they are using these kinds of tools in the hiring process.

AI audits have become a growing compliance priority across the world, not just in New York.

It won't go into effect until next year, but the European Union passed a much broader law last year that requires "Very Large Online Platforms and Search Engines" (I had to look up Zalando) to disclose how they use algorithms to make a wide array of decisions or recommendations.
Former Protocol Enterprise AI whiz Kate Kaye reported last year on the growing number of startups getting into the AI audit business in response.
A lot of companies working on the types of hiring tools covered by the New York law, such as Hired.com, have gotten ahead of the curve by conducting their own audits, but New York's law requires an independent auditor.

Backers of the law hope that by forcing companies to disclose how the software they use makes hiring decisions — assuming they need to hire people living in the biggest city in the U.S., which includes almost all major enterprise companies — they'll demand software that reduces or eliminates bias.

There is no guarantee that will actually happen.
But it's not hard to imagine businesses will begin to rely more heavily on automated software tools to make hiring decisions as they look to squeeze more efficiency out of their HR departments, especially in these generative AI boom times.
Ideally, the vendors that make those tools will now have a strong incentive to reduce the bias inherent in their algorithms if they start losing business due to the results of these audits.
Of course, many of those same companies lobbied heavily to reduce the disclosure requirements of New York's law.

The real-world effects of New York's law will be watched very closely by enterprise software companies, even ones that aren't in the HR business.

Dealing with compliance requirements is going to become a fact of life over the rest of this decade, given how important software has become to the world.
The capabilities of that software have evolved much faster than approaches to regulate it, thanks to the usual grumbling from Silicon Valley policy teams that accompanies any attempt to erect a speed bump.
But there's an upside: every business of a certain size will need software to help them meet these compliance requirements.
That could be an enormous opportunity for existing enterprise software companies or startups to offer reliable and easy-to-use compliance software and services to help customers deal with that new reality.

Year of the AI prompt

China's two biggest cloud companies unveiled their latest approaches to generative AI Friday, right as China's AI sector starts to grapple with the long-term effects of export controls on powerful GPUs.

Alibaba released Tongyi Wanxiang, a generative AI tool that can create images based on text prompts, similar to OpenAI's DALL-E. And Huawei introduced a new version of its large-language model designed specifically for enterprise business cases.

It will be a challenge for both companies to obtain the computing power necessary to train future versions of those products, thanks to restrictions on the sale of Nvidia's most powerful chips imposed by the U.S. government. But they also face restrictions on the types of generative AI tools they can build thanks to Chinese data regulations, which are much more comprehensive than anything found in the U.S.

Quote of the week

"I used to tell people, 'Hey, tidy your workspace. Keep everything organized. And now with AI, I'm sort of like, 'It actually doesn't matter how you kept it, just throw everything in there and we will make sense of it." — Akshay Kothari, co-founder and COO of Notion, on the future of project management tools.

The Runtime roundup

Progress Software released patches for three new vulnerabilities it discovered in its now-infamous MOVEit file-transfer software, which contained previous flaws that were responsible for one of the biggest data breaches in modern history.

Cloud services companies recorded $545.8 billion in revenue during 2022, a 23% increase despite a worldwide slowdown in enterprise tech spending after the invasion of Ukraine and rising interest rates.

Revenue from SUSE's core enterprise Linux product fell 1% during the last quarter, and it cited longer sales cycles and economic unease for the shortfall.

Here are two great weekend reads: The Wall Street Journal profiled Atsuyoshi Koike, a septuagenarian entrepreneur and football fan hoping to lead a semiconductor resurgence in Japan; and TechFinitive took a look at how AI is helping logistics companies put an end to the snarled supply chains that governed so much of life during the height of the pandemic.

Thanks for reading — see you Tuesday! And please help me with an experiment: click here if you read this far!