Illustration: Nadia Méndez/WIRED Middle East

Ever since the name ChatGPT came to public attention, productivity in the workforce has taken a great leap forward. Every industry in the world has rushed in to take a share of the artificial intelligence (AI) cake, and Wall Street is no exception.

In 2023, a frequently cited study from the University of Florida hinted that a ChatGPT trading algorithm could deliver 500 percent returns in the stock market, outpacing conventional sentiment analysis models used by most hedge funds. Myriad AI-based trading tools soon appeared across the market.

Bloomberg News reported last year that at some of the most enthusiastic banks on Wall Street, 40 percent of their job hires are for AI-related talents. These roles include data engineers, quantitative analysts, and ethics and governance roles, according to data from the consultancy firm Evident.

Evident’s data showed that the biggest US bank, J.P. Morgan, listed 3,651 AI-related roles globally from February through April in 2023, almost double those of its closest rivals Citigroup and Deutsche Bank. Eigen Technologies, a research-driven AI company that helps firms including Goldman Sachs and ING with AI, reported a five-fold jump in inquiries from banks in the first quarter of 2023 compared to the same period a year ago.

A report by Deloitte also shows that the top 14 global investment banks have been able to boost their front-office productivity by as much as 35 percent with generative AI. This could result in additional revenue of $3.5 million per employee by 2026.

The AI craze on Wall Street is growing by the day and trading with AI seems to be the trend now.

Predictions in an unpredictable market

AI in the finance sector involves a wide range of models that go beyond generative AI like ChatGPT. The most prominent characteristic of AI is its ability to process a massive amount of data in a short period of time. In stock trading, analysts and traders often have to look for signals from an overwhelming sea of information. And AI can easily gather that information, from historical price trends to news and social media sentiments.

“Take investor sentiment, for example,” says Shawn Edwards, Chief Technology Officer at Bloomberg. “One way to measure sentiment in the stock market is to analyze news, research, and social media posts. The unstructured nature of textual information presents challenges that are comfortably addressed by machine learning techniques to increase confidence in text-based signals.” Data science-empowered technologies enable real-time observations from sources like consumer credit card transactions, satellite images, shipping information, supply chain data, earning call transcripts, and other economic indicators.

Gathering information is only the first step, organizing and analyzing the data via complex algorithms are the key if AI is to make predictions. “When these observations are linked to traditional market information, never-before-seen relationships can be discovered, helping users make more informed decisions,” Edwards adds.

The power to quickly extract useful information from massive amounts of data might make the stock market seem less volatile for AI than for humans. “The sudden unpredictable market might be abnormal for humans, but rarely [have] these moves not happened in history,” says Dr. Rein Wu, the ex-Visa researcher who founded the AI-enabled prediction market PredX. “We can use historical patterns to match with current stock patterns, including sudden moves, and project what happened next with a probability distribution.”

Natural Language Processing (NLP) and machine learning (ML) hold the key to this data–gathering, organizing, and analyzing process. Dr. Rein named Long Short–Term Memory (LSTM) as proof of a highly effective algorithm that is used in trading. It has an exceptional ability to capture and make good use of the data, he says, “Similarly, there are Random Forests, a commonly–used machine learning algorithm that combines predictions from multiple models, thereby providing robustness,” he adds.

Several models built on these algorithms, such as Recurrent Neural Networks, Gradient Boosting Machines, AdaBoost, Convolutional Neural Networks, Deep Q–Networks, and Deep Deterministic Policy Gradient, were used to predict stock prices and have proved to be quite effective, according to Dr. Rein.

Illustration: Nadia Méndez/WIRED Middle East

Trading with AI

“AI has certainly helped make my trading better, bridging the gap where my biases have cost me some serious profits,” Dr. Rein says. Even for an experienced and knowledgeable investor like himself, human bias and sentiments inevitably “mar the trading experience and the bank balance,” he says. With AI, these undesirable elements can be easily removed. However, he also admits that human-monitored strategies are still crucial in trading.

As a company that has been building and deploying AI for more than 15 years, Bloomberg has received all sorts of feedback from its customers who have taken advantage of its AI solutions. The company has incorporated generative AI into its products and services, launching its large language model specifically for finance, BloombergGPT, and the AI-powered Earning Calls Summaries for its customers, researchers, and analysts.

Its earnings transcript summarization system can quickly decipher complex financial information, extract key insights, and produce summarized answers on topics like capital allocation, hiring plans, supply chain issues, and more. Normalizing and enhancing data with AI enriches the extracted information, making it more accessible for those who need it. “Each day, Bloomberg processes millions of news articles, research documents, and social media posts, annotating them with metadata such as entity and topic identifiers, sentiment analysis, importance assessment, and salience ranking,” explains Edwards. NLP techniques help the company construct a knowledge graph encompassing companies, individuals, products, events, actions, and topics.

Meanwhile, BloombergGPT excels at answering conversational finance questions—a complex task that was previously done by human employees with rigorous numerical reasoning. “It requires an understanding of structured data and financial concepts, and the need to relate follow–up questions to the turns in dialog about S&P 500 earnings reports that include text and at least one table with financial data,” says Gary Kazantsev, Head of Quant Technology Strategy at Bloomberg.

Apart from data gathering, extracting information, and conversational tasks, the use of AI in trading has also enabled traders to make better decisions by analyzing vast amounts of data quickly and accurately. It has enabled some traders to automate their trading strategies and take advantage of market opportunities 24/7 with High-Frequency Trading (HFT), a type of algorithmic trading that involves executing trades at very high speeds—sometimes even in fractions of a second.

However, one of the major limits of AI trading is its inability to predict black swan events and extreme market conditions. While AI models can be trained on historical data, they are not able to predict sudden market changes or unexpected events that can significantly affect the market.

AI trading could also potentially lead to amplification of the market. Despite AI algorithms’ quick responses to market changes based on real-time data, they might also contribute to market volatility as they may all respond to the same market indicators simultaneously.

Some researchers have noticed that the effectiveness of algorithmic trading in the short term is heavily influenced by news more than other indicators. In a personal project, Hasan Mustafa Hosny, an Automation Engineer at the Swiss Stock Exchange experimented using the LSTM algorithm over a period of three weeks. “Intraday trading carries a high level of risk,” Hosny says. He also points out that the stock market is very unpredictable in the short term with breaking news kicking in, influencing the trend and prices. “Plus there is a lot of price manipulation done by hedge funds and traders, so the price movements do not always tell the truth about the market, especially with AI and HFT affecting the price movements,” he adds. “In the short term, the sentiment of the market and news has a huge influence on the stock movement, and usually gives great results in return, even if they are not really based on facts or other data,” he concludes. To further study breaking news’ impact on ongoing trades, Hosny is creating a new algorithmic trading project that would run 24/7, scanning news from all over the world, with the algorithm designed to autonomously decide which securities to invest in based on the news.

Many industry experts arrive at the same conclusion, that the human factor is still crucial in trading as AI still has a lot of limitations. “I think that for the foreseeable future, humans will continue to play a significant role in the investment process, especially considering some of the constraints, such as the finite amount of computing available,” Kazantsev says.

“AI’s performance is largely dependent on the quality of its training data,” adding to the limitations of AI, Hosny says. “The data we feed into the AI model is a crucial factor in determining its performance.” As for now, it’s more sensible to look at algorithmic trading as an analytical assistant that can monitor market trends. The task of deciphering market nuances still largely depends on human expertise.

Will we see an AI-powered scandal on Wall Street?

Illustration: Nadia Méndez/WIRED Middle East

AI is ubiquitous on Wall Street. Quantitative hedge funds and leading investment firms like BlackRock and J.P. Morgan are already using AI, with many following across the financial markets. The SEC (US Securities and Exchange Commission) has also greenlit Nasdaq’s AI trading system, which utilizes reinforcement learning (RL) algorithms to make real-time adjustments. The threat of supercharged AI manipulation of the stock market seems to be lurking around the corner.

“Informed AI traders can collude and generate substantial profits by strategically manipulating low order flows, even without explicit coordination that violates antitrust regulations,” warned a research paper by finance professors Winston Wei Dou and Itay Goldstein from the Wharton School at the University of Pennsylvania. This might be done either through a “price-trigger mechanism” designed to penalize deviations in trading behavior or through the cultivation of uniform learning biases among algorithms, according to the paper.

“When these algorithms are programmed with the intention of influencing the market, they can enhance the ferocity of certain movements, causing great market volatility,” says Dr. Rein. “There’s even a possibility that the algorithm uses market manipulation as an investment strategy.”

“As the adoption of traders’ AI accelerates, a balance must be maintained to ensure not just fairness but also to enhance trading proficiency and mitigate its impact on the entire financial system,” he adds.

Meanwhile, as many researchers point out, the quality of data could further complicate the matter. The available datasets in stock markets may be rife with errors and inconsistencies. Such inaccurate, incomplete, or biased data inevitably deteriorate the models, consequently offering deceptive insights and potentially leading to expensive trading judgments.

“Most institutions are taking a very careful approach to using them, especially due to the non-public data they’re being used to analyze,” Kazantsev noticed. Adding to that are the current models’ other limitations, such as hallucinations, a lack of reasoning, and the inability to perform advanced mathematical tasks.

Regulators are also alert to the potential threat of AI in financial markets. Gary Gensler, the chair of SEC, recently warned against “the possibility of AI destabilizing the global financial market if big tech-based trading companies monopolize AI development and applications within the financial sector.”

Scrutiny is also coming from regulators around the globe. The European Union has already introduced the AI Act, while the United Nations has adopted the first global AI resolution. Dr. Rein points out that despite the absence of an overarching AI legal framework in the Middle East, “concerns regarding market manipulation, fairness of trading practices, and transparency in data usage are raising some serious questions for regulators, who have begun to take notice.”

“However, it is important they strike a balance to accommodate advancements in technology while protecting the market,” he adds.

The AI transformation is unstoppable

Stock market trading is already undergoing a significant transformation amidst the AI revolution. “But it is just the beginning. As the technology advances, especially with artificial general intelligence (AGI) looking like a real possibility in the not-so-distant future, I believe it is going to get even crazier,” says Dr. Rein.

For starters, AI’s use in algorithmic trading will allow users to capitalize on miniscule price fluctuations that humans fail to process. AI would likely play a key role in risk management, reducing potential losses while providing more personalized investment strategies to amplify profits. Furthermore, market predictions and forecasting could be more accurate, which would enable humans to make better and more informed decisions.

“I think we can easily anticipate the industry rapidly exploring the capabilities and limitations of AI over the next year and focusing on key use cases to improve the efficiency of various automatable or augmentable workflows,” says Kazanstev. Some examples include reading and summarizing complex documents, improved search and analytics, and code generation.

Kazanstev also points out that the adoption of AI will speed up considerably when users are able to run a state-of-the-art LLM on their laptops. “This technology will then be incredibly transformative,” he says.

“I envision a day not too far in the future when one of these models could power a personalized assistant that remembers details about you and what you’re working on, and that can help you with complex tasks.”

As AI continues to disrupt Wall Street, it seems that human wisdom is the bridge between AI’s power and limitations and it will be some time before that can be removed from the equation. 

• Share