Algorithmic trading uses computer programs and automated instructions for trade execution. Also known as algo trading or black-box trading, it has captured over 50% of the trading volume in US markets today.
Here’s a fascinating account of how algorithmic trading has evolved through phases and gained prominence recently.
Securities trading has undergone a profound metamorphosis over the last few decades. Today, “Algorithmic Trading” or “Algo Trading” has taken over, and exchanges execute orders backed by automated and pre-programmed instructions.
Institutional investors and larger brokerages use algorithmic trading to reduce expenses and manage large orders. Even mutual funds, pensions, and other large-scale investment vehicles use it. Typically market-making players use algorithmic trades to create liquidity. In Europe and the US, 10% of the hedge funds used algorithmic trading for over 80% of their value in 2020.
Large institutions have also gone a step ahead and began using supercomputers to trade within nanoseconds. This is a relatively new phenomenon in the algorithmic trading landscape called high-frequency trading. After the US Securities and Exchange Commission (SEC) allowed ECNs in equities trading in 1998, more algo traders entered the space. In 2000, the SEC enacted decimalization, and many players earning through spreads were compelled to switch to high-frequency trading for a larger trading volume. High-speed trading has increased exponentially over the last decade. This activity comprises a large percentage of the total trading volume.
HFT does contribute to market efficiency, but it could also lead to market volatility, especially during stressful times. Securities and Exchange Commission and the Commodity Futures Trading Commission indicated that algorithmic trading and High-Frequency Trading contributed to volatility in 2010.
In this article, we have put together an intriguing history of Algorithmic Trading from the days of carrier pigeon to the current phase of High-Frequency Trading.
17th Century: Rothschild Family Used Carrier Pigeons For Information Arbitrage
Credit goes to Nathan Mayer Rothschild, a German financier, and his family for using the earliest form of information arbitrage.1 In the early 1800s, news traveled at a languid pace. There were a handful of organized courier services, and the telegraph was at a conceptual stage. Rothschild used carrier pigeons to arbitrage prices of the same security by relaying the information before the computers did it.
It was only due to these pigeon carriers that Rothschild was the first in London to know about England defeating France at Waterloo.2 While other traders were preparing themselves for losses, Rothschild took an extended position. This was not just an isolated incident. Nathan, who stayed in London, regularly received market insights about Frankfurt, Paris, Naples, and Vienna through the pigeon carriers.
Acting upon the news piece received through carrier pigeons was the earliest form of arbitrage trading.
1832: Samuel Morse’s Telegraph Facilitated Financial Communication
Developed in 1832, Samuel Morse’s “the telegraph” was a pathbreaking invention in the history of long-distance communications.3 The telegraph transmitted electrical signals through the connecting wire between stations. In addition to inventing the telegraph, Samuel Morse also developed a code- the Morse code comprising dots and dashes, for transmitting complex messages across telegraph lines.
In a few years, even the trading industry started using it.4 The telegraph helped to distribute the Financial newsletters in areas far away from the marketplace. It also lowered information costs inside and between firms. By 1856, the telegraph also facilitated broker-assisted trading of exchange-based securities.
1850: News Services- Reuters Was Established
Julius Reuter, the founder of Thomson Reuters, employed various technologies, including telegraph cables alongside a fleet of carrier pigeons to run a news delivery system.5 In April 1850, Reuter formed an agreement with Heinrich Geller to launch a news transmission service between Aachen and Belgium using carrier pigeons. The service also aimed to relay stock price information. The service continued for a year, after which the telegraphic gap between both nations closed.
Julius Reuter established a telegraph office on October 10, 1851, at the I Royal Exchange Buildings, close to the London stock exchange.6 Reuter used this location to relay stock market quotations between London and Paris. For this, he used the new Calais-Dover telegraph cable under the English Channel. A few years later, in 1858, he successfully convinced the London Times and other English dailies to subscribe to his new service, which he named Reuters.
1867: First Stock Ticker Developed By Calahan
In 1867, an engineer at American Telegraph company, Edward A. Calahan, upgraded a telegraph machine for printing financial data specially tailored to the stock market.7 This invention saw a lot of success, and the word spread around quickly. The device offered stockbrokers and traders a chance to access recent stock price information without any physical presence at a stock exchange. Exchanges could transmit the name and price of every stock sold telegraphically. The stock ticker’s capacity for broader communications allowed the NYSE to centralize the order flow and improve liquidity.
Just when Calahan had finished assembling his fourth machine, he received orders for many more batches.8 These devices were eventually known as stock tickers, as their printing wheels made ticking sounds like clocks. One of the earliest demonstrations of the stock ticker was at a brokerage office of David Groesbeck & Company. Later, even Gold and Stock Telegraph Company in New York became the buyer of Calahan’s stock ticker.
1871: Edison Develops Stock Ticker
Thomas Alva Edison substantially improved Calahan’s invention and designed a new stock ticker called the Edison Universal Stock Printer in 1971.9 It was a Brass mechanism with a cast-iron base and glass dome. It measured 330 x 103mm, featuring a blank roll of ticker paper and a base that said: “Quotations furnished by the Western Union Telegraph Co / Apply to local manager.”
Interestingly, the Gold and Stock Telegraph Company was also a client of Edison’s stock ticker mechanism. This ticker was the first mechanical mode for transmitting real-time stock market data directly from exchange floors to intermediaries and investors throughout the country. The most significant improvement from Calahan’s model was this mechanism’s ability to activate all tickers on a particular to work in tandem and print the same information.
1929: The ‘Big Board’ Was Installed
In 1929 the Teleregister Corporation installed the first stock information display board. This automated quotation board, also known as the ‘Big Board’, was a large vertical electronic display located in an exchange.10 Many brokerage offices also displayed the information related to the stocks chosen by the broker. The information also included the current price, opening price, previous closing price, high for the day, low for the day.
Some of the exchanges from which the Teleregister board offered data include the New York, American, Chicago Mercantile, New York Cocoa, New York Coffee andSugar, New York Mercantile, New York Produce, New York Cotton, and New Orleans Cotton exchanges, and the Chicago Board of Trade. Some firms had a line of telephone operators that sat in front of a Teleregister board to relay price and trading volume data to the brokerage houses.
This electronic quote display board laid the foundation of the “open outcry” trading system whereby traders met in a common location or exchange floor and made transactions based on standard price and volume information vision the Big Board.
1930: Western Union Launches 5-A
Stock tickers were the need of the hour, but the existing ticker systems couldn’t keep pace during a market crash. After the stock market debacle of 1929, the Western Union launched its Ticker-5A, a 500-character-per-minute “black box” processing up to 8 million trades a day.11
1949: Richard Donchian Launched The First Trading Rule-Based Fund
In 1949, Richard Davoud Donchian, an American trader, launched Futures, Inc., a publicly-held commodity fund trading the futures markets.28 It was unique because that fund was one of the first to use a set of predetermined rules to generate actual trading buy and sell signals. We can call it the earliest form of an automated trading system.
Donchian used a mathematical system based on moving averages of commodity market prices. It functioned very differently back then without any internet to support it. The developers had to depend on ticker tape while they charted individual markets manually. It was a cumbersome process.
1952: The Markowitz Model
In the 1950s, Harry Max Markowitz, an American economist, introduced computational finance to address the issue of portfolio selection.12 Markowitz introduced the Modern Portfolio Theory or MPT to academicians in his article, “Portfolio Selection,” which was featured in The Journal of Finance in 1952.
Markowitz established the correlations between securities and diversification through his theories and emphasized the criticality of portfolios and risk. Markowitz is also known as the father of quantitative analysis.
Markowitz’s theory is one of the most well-known portfolio-allocation approaches used in present-day algorithmic or quantitative trading. The objective behind the Modern Portfolio Theory is to calculate the covariance between the returns of all the trading strategies in a portfolio and use it to design risk allocation for individual trading strategies to limit portfolio variance and maximize portfolio returns. However, Markowitz’s theory and analysis faced a significant problem because of the shortage of computer power.
1960: The First Arbitrage Trade Using Computers
In the 1960s, hedge fund managers Ed Thorp and Michael Goodkin partnered with Harry Markowitz and became the first traders to employ computers for arbitrage trading.13 After introducing personal computers in the late 1970s and early 1980s, many computational finance applications were also launched. Signal processing led to several new techniques instead of the conventional forms of economic models, such as time series analysis and optimization.
1960: Quotron I And Ultronics Were Launched
In the early 1960s, two computerized stock quote delivery systems known as Ultronics and Scantlin Electronics were launched.14 These systems show the recent price and the upside or downside trend of the bid/ask spread. Jack Scantlin of Scantlin Electronics, Inc. (SEI) developed the Quotron I system that featured a magnetic tape desktop storage unit with a keyboard and printer. The data was recorded into the storage unit from the ticker line.
Robert S. Sinn formed Ultronic Systems Corp. in December 1960. The first desk units of Ultronics, also called the Stockmaster, were installed in New York, Philadelphia, San Francisco, and Los Angeles. Like Quotron I, these desk units also constantly monitored the last sale, bid, ask, high, low, total volume, open, close, earnings, and dividends for every stock on the NYSE and the AMEX.15 It also offered quick access to the commodities market data from several US commodity exchanges.
1965: NYSE’s MDS I and MDS-II
Since 1965, the New York Stock Exchange’s trade reporting system, the Market Data System I, provided automated quotes.16 MDS I could provide over 100,000 trades a day and had seen a downtime of fewer than 21 minutes per year. Hence, it was also very reliable. The success of MDS I led the NYSE to develop the MDS-II, an improved system fully operational by July 1972. MDS-II was created to improve reliability and boost the capacity of the trade reporting system by three times.
1967: Instinet Trading System Developed
In 1967, Instinet was founded by Jerome M. Pustilnik and Herbert R. Behrens and incorporated as Institutional Networks Corp. Instinet is the oldest electronic communications network on Wall Street. Instinet was positioned as a competitor with the New York Stock Exchange while communicating through computer links between banks, mutual fund houses, insurance companies without any delay or intervention.17
The introduction of the Instinet enabled large institutional investors to trade pink sheet or over-the-counter securities directly with one another in an electronic set-up.
Over the years, Instinet has emerged as a powerful competitor to traditional exchanges. Along with sales trading services, Instinet also provides advanced trading technologies such as the Newport EMS, trade cost analytics, commission management, algorithms, independent research, and dark pools of liquidity. Instinet is best known as the first off-exchange trading alternative in the world. In the 1980s and 1990s, its “green screen” terminals became more widely known.
1971: The Formation Of Nasdaq
With the formation of the National Association of Securities Dealers Automated Quotations (NASDAQ) in 1971, fully automated over-the-counter (OTC) trading turned into reality.18 Initially, NASDAQ only offered quotations, but eventually, it provided electronic trading.NASDAQ is the second oldest stock exchange in the United States and the first to offer online trading.
Just like Instinet, NASDAQ used advanced information systems and connectivity technology for its operation. Over the years, it became more of a stock market as it added the reporting of trading volume in its automated trading systems.
1978: Intermarket Trading System
NASDAQ Intermarket Trading System was a crucial participant in the technology race as traders rushed to establish computers to boost electronic trading speed. This network leveraged the Intermarket Trading System (ITS), an electronic network that linked the trading floors of various exchanges and allowed real-time communication and trading between them.19
Through the ITS platform, any broker on the floor of a participating exchange could respond to real-time price changes and place an order under coordination. The Securities Industry Automation Corporation (SIAC) manages the ITS.
1981: The Launch of Bloomberg
In 1981, Michael Bloomberg, general partner of Salomon Brothers, was offered $10 million as a partnership settlement for designing in-house computerized financial systems.20 Bloomberg constructed the Innovative Market Systems (IMS). IMS received an investment from Merrill Lynch worth $30 million to develop the Bloomberg terminal computer system. By 1984, all Merrill Lynch clients were buying the IMS machines. Initially, the company sold computer terminals to Wall Street investment banks that contained financial data about stocks, bonds, and other investments.21
In 1990 Bloomberg Business News was launched. The news service was provided on the company’s computer terminals. By 1991, Bloomberg Business News bureaus were operational in New York, Washington, London, Tokyo, Toronto, and New Jersey, and in 1994, it launched its business news channel, Bloomberg Television.
Bloomberg brought about a path-breaking innovation in the field of real-time information about forty years ago. It develops highly advanced technology to collect and monitor massive amounts of data that it offers its subscribers.
1982: The Launch of Renaissance Technologies
In 1982, Jim Simons, one of the greatest American investors, founded Renaissance Technologies, a quant fund. 31 Renaissance leveraged mathematical models to predict price fluctuations of financial instruments. These models analyzed maximum data and looked for non-random price changes to make trading decisions.
Renaissance Technologies had gained prominence for its Medallion Fund, which was based on a $10 billion black-box algorithmic program trading. The fund was only for Renaissance owners and employees, excluding average investors.
By 1988, Simons only used quantitative analysis to pick the trades to enter. Expert mathematicians, data analysts, cryptographers, and physicists combined with sophisticated technology and high-frequency strategies were integral parts of the funds. The company thrived due to the complex mathematical formulas these scientists developed.
1984: Dataspeed Launched Quotrek
In 1984, Burlingame, California company Dataspeed, Inc. launched the QuoTrek, a wireless mobile stock quotation delivery system in Los Angeles, Chicago, San Francisco, and New York.15 Dataspeed felt that it would receive an overwhelming response like the ticker tape. But that did not happen.
Even after winning some significant contracts, it didn’t make a breakthrough because brokers already had access to such information. Also, the process for setting up the system was quite cumbersome. Dataspeed had to approach radio station managers to access the station’s transmitters to install their QuoTrek transmission equipment.
1984: NYSE Computerized Order Flow
In the early 1970s, computerization of the order flow began in the financial markets. In the mid-1970s, NYSE chairman William (Mil) Batten and vice-chairman John Phelan began persuading the exchange to be more technology-oriented.
The landmark event, in this case, was when the New York Stock Exchange launched the “designated order turnaround” system (DOT, and later SuperDOT) in 1984.22 The DoT routed orders electronically to the proper trading post, which executed them manually. The “opening automated reporting system” (OARS) aided the specialist in determining the market clearing opening price (SOR; Smart Order Routing).
SuperDOT facilitated a market order transmission from a member firm directly to the NYSE trading floor for execution. Once the order was executed on the floor, the firm got an order confirmation. The SuperDOT system marked a significant step in equities trade execution both in terms of speed and volume. It allowed orders up to 2,000 shares that could be electronically routed to a specialist.
1993: Interactive Brokers was founded
A pioneer in digital trading, Thomas Peterffy, founded Interactive Brokers in 1993. The objective behind Interactive Brokers was to popularize the technology that Timber Hill developed for electronic network and trade execution services to customers. Timber Hill created the first handheld computer for trading.
On the other hand, Thomas Peterffy created the first fully automated algorithmic trading system in 1987. The system consisted of an IBM computer that could extract data from a connected Nasdaq terminal and carry out trades fully automated.37
1996: Launch of Island-an ECN
One of the most successful Datek traders, who also created the Watcher, designed a system that facilitated users to trade between themselves without interacting with market makers.23 This electronic communication network (ECN) known as “Island” was finally launched in 1996.
Island was an innovative movement as traders could receive information on stocks through an electronic feed once they subscribed to their services. To promote the process and encourage trading, traders were reported to a system that contained a program constantly providing real-time execution price and volume information.
1998: The US Sec Allowed Alternative Trading Systems-Reg ATS
In 1998 US Securities and Exchange Commission (SEC) allowed electronic exchanges and paved the way for computerized High-Frequency Trading.24 The SEC adopted new rules and rule amendments to allow alternative trading systems to decide whether to register as national securities exchanges or register as broker-dealers and comply with additional requirements under Regulation ATS, depending on their activities and trading volume.
Being regulated added credibility and transparency to high-frequency trading and boosted its growth. By the end of 2009, high-frequency traders had captured 70% of the US securities markets.32
2001: US Decimalization Process Completion
In the late 1980s and 1990s, fully electronic execution and similar electronic communication networks developed. The change in the minimum tick size from 1/16 of a dollar (US$0.0625) to US$0.01 per share changed the market structure by permitting more minor differences between the bid and offer prices. This process known as the U.S Decimalization has encouraged algorithmic trading immensely.25
The implementation period for this (the “Phase-In Period”) began on August 28, 2000, and the decimal pricing for all equities and options ended by April 9, 2001. Finally, the New York Stock Exchange and the American Stock Exchange switched to decimalization on Jan. 29, 2001. After 2001, all price quotes have always been expressed in the decimal trading format instead of fractions in price quotes.
The switch to decimalization was done to comply with standard international trading practices. It also became simpler for investors to identify changing price quotes and respond to them. Decimalization has resulted in tighter spreads due to the smaller incremental movements of the price.
2005: Implementation of the Reg NMS
Formulated in 2005 and established in 2007, the Regulation National Market System (Reg NMS) was a series of initiatives to modernize and strengthen the national market system for equities.31 This altered the way firms that operate on the Trade Through Rule basis.
The Reg NMS’s Order Protection Rule mandated that exchanges transmit real-time data to a centralized entity, consolidating the information into a unified data feed, requiring that exchanges and brokers accept the most competitive offer when matching buyers and sellers. Many experts believe that it offered an advantage to high-speed trades, which had become a dominant phenomenon.
2008: The Creation of Pandas at AQR
In 2008, Wes McKinney, a developer at AQR Capital Management, created Pandas, a data manipulation and analysis software package for the Python computer language.32 It was made due to the demand for a high-performance, versatile tool for performing quantitative research on financial data. In 2012, Chang She, another AQR employee, joined him in his efforts. Pandas is mainly used to analyze data.
Pandas support data input from various file formats, including comma-separated-values, JSON, SQL, and Microsoft Excel. Pandas also support a variety of data manipulation operations that includes merging, reshaping, and selecting. It is also used for data cleaning and wrangling.
2010: Spread Networks Launched Fastest Dark Fiber Services
In August 2010, Spread Networks, a private company, launched the fastest and most trustworthy dark fiber services between the greater New York and greater Chicago metropolitan areas.29 At 825 fiber miles and 13.3 milliseconds, the route of the Spread Networks was the shortest and most diverse between New York and Chicago.
Spread Networks markets its bandwidth and dark fiber to wholesale carriers, enterprises, and financial firms with premium low-latency and competitive wholesale services. The appeal of the Spread network lies in its route, which is almost a straight line from a data center in Chicago’s South Loop to a building across the road from Nasdaq’s servers in Carteret, N.J. Some of the earlier routes largely follow the railroad through Indiana, Ohio, and Pennsylvania.
2010: The Flash Crash
One of the by-products of the rise in algorithmic trading using computer algorithms is flash crashes. Flash Crashes are a semi-regular occurrence, sometimes impacting just a single stock or a broader market.33 The first Flash Crash was in 2010. Algorithmic and high-frequency trading caused extreme market volatility resulting in a flash crash in the stock market on May 6, 2010. The Dow Jones Industrial Average plunged about 600 points but recovered its market value within minutes.
2011: Fixnetix Launches A Microchip Powered By Nanotechnology
High-Frequency Trading has an execution time of several seconds that shrunk to milliseconds and even microseconds by 2010. Subsequently, it was reduced to nanoseconds in 2011. Nano trading technology was launched in 2011 when a London-based trading technology company, Fixnetix, developed a microchip that had the power to execute trades in nanoseconds.30
Called iX-eCute, it was an FPGA (Field Programmable Gate Array) microchip for ultra-low latency trading. Low latency trading includes algorithmic trading systems and network routes that financial institutions use while connecting to stock exchanges and electronic communication networks (ECNs) for executing the transactions rapidly. The microchip could conduct 20+ pre-risk checks taking less than 100 nanoseconds.
2011: Quantopian
John Fawcett and Jean Bredeche founded Quantopian in 2011. The company’s business model was two-pronged:34 The first side focussed on algorithm-developers who developed their own trading algorithms and tested for free. The objective was to create computerized trading systems and strategies for elements that Quantopian could add to its offerings to institutional investors.
It organized contests called “Quantopian Open”, where participation was open irrespective of education or work background. Quantopian offered accessible data sources and tools, primarily built in Python.
The second aspect of Quantopian focussed on institutional investors. The investments of these members had their assets managed by the winning algorithms. Successful developer-members were entitled to a royalty or commission from these investor-members, who earned profit from their strategies.
In November 2020, Quantopian was declared insolvent and was shut by the authorities.
2012: Dataminr Turns Tweets Into Actionable Trading Signals
In September 2012, Dataminr, a New York-based start-up, launched a new service that turned tweets and other social media communications into actionable trading signals.26 This would boost the appearance of the latest news 54 minutes faster than the traditional news coverage. The platform could identify a lot of “micro-trends,” which provided clients with unique insights and helped them foresee the upcoming trends.
2014: Michael Lewis Publishes The Book Flash Boys
Algorithmic trading and high-frequency trading garnered public attention after author Michael Lewis published his book Flash Boys in 2014, which eventually became a best seller.27 The book presented an account of the lives of Wall Street traders and entrepreneurs who together laid the foundation of electronic trading in the US.
His book also shed light on the rat race among the tech companies to build faster computers to enhance the speed of information transfer between exchanges. The book also focuses on the darker side of electronic and high-frequency trading, making Flash Boys quite an intriguing read. It also exposes the many different ways through which Wall Street players earn profits.
2015: CFTC Approved Proposed Rule on Automated Trading
The US Commodity Futures Trading Commission (Commission) approved proposed rules for increased regulation of automated trading on US designated contract markets (DCMs).38 These proposed rules represented a series of risk controls, transparency measures for algorithmic trading.
The objective of the proposed rules was to reduce potential risks from algorithmic trading activity. The plan of action involved implementing risk controls through maximum order message, maximum order size parameters, and establishing standards for developing, testing, and monitoring of ATSs.
2020: COVID-19 and Algorithmic Trading
The trying times of the Covid-19 pandemic gave an impetus for human traders to implement computerized trading systems. Traders opted for algorithmic trading strategies for lower human errors and faster decision-making during times of uncertainty such as the pandemic. Market participants turned to tested Algo trading strategies during periods of volatility and reduced market liquidity. According to a survey, the proportion of participants trading 80% or more of their portfolio through algo trading market share almost doubled to 20.75% in 2021 from 10.98% in 2020.36
Studies have confirmed that stocks with the highest algorithmic trading witnessed lesser liquidity reduction and demonstrated more market stability than stocks with lower algorithmic trading activity.36 This indicated that Algorithmic Trading didn’t lead to liquidity depletion from the market during times of crisis and high volatility.
The Bottom Line
When trading algorithms were first introduced, they were mainly designed for basic arbitrage, pairs trading, market-making strategies, and automation. It has evolved over the years in phases. In the next phase, algorithmic traders pursued quantitative analysis using real-time data from the order book to create trading models using volume-weighted average price and other simple investment strategies. Traders also analyzed how the model could be adjusted according to changing market conditions. Currently, development centers around the integration of machine learning and related technological advances into algorithmic strategies.
Sources: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40
