Norgate Data Python Tutorial

Attention algorithmic traders and quantitative analysts! Are you tired of manually collecting and analyzing market data? Do you want to automate your end-of-day trading activities using a powerful and customizable Python application? Look no further than Norgate Data. With Norgate Data’s Python plugin, you can easily integrate Norgate Data’s high-quality financial data into your Python trading system and streamline your analysis and execution process.

 As a seasoned algorithmic trader and Python developer, I can provide the insights and guidance you need to make the most of this powerful tool. So why wait? Start automating your end-of-day trading activities with Norgate Data’s Python Plugin and unlock your trading system’s full potential today!

What is Norgate Data?

Norgate Data is a company that provides historical and real-time financial market data to traders, investors, and analysts. The company was founded in Australia in 1991 and has since become a trusted source for end-of-day data, including equities, futures, and options data from major global exchanges.

Norgate Data’s Python plugin is particularly popular among algorithmic traders and quantitative analysts because it allows them to access and automate their end-of-day trading activities using Python applications. Whether you are a beginner or an experienced trader, Norgate Data can help you make better-informed investment decisions.

Norgate Data Pricing Plans

Norgate data price plans depend on three factors:

  1. The time period of the subscription, which can be 6 or 12 months.
  2. The type of the data you want to retrieve e.g, US Stocks Daily Data, Australian Stocks Daily Data, futures daily data etc.
  3. The subscription tier i.e., silver, gold, platinum, or diamond.

You can check the official website to customize your price plans.

Alternatively, Norgate Data offers a free 3 weeks trial. I strongly recommend you sign-up for the trial and evaluate the product before signing up for any paid subscription.  

Norgate Data Pros and Cons

Pros:

  • Provides survivorship bias-free data from multiple international markets.
  • Official and third-party plugins for various platforms.
  • Integration with backtesting and deep learning packages.

Cons:

  • Very high price.
  • No REST API. You need to install a data updater.
  • Python plugin only for Windows platform.

Setting Up Norgate Data Python Plugin

This section explains how to install the Norgate Data plugin for Python. I will explain how to sign-up for the Norgate Data trial subscription, install the data updater and fetch data using Python code.

Requirements

Per official documentation, you need the following to run the Norgate Data Python plugin.

  1. Python 3.5+.
  2. Microsoft Windows.
  3. Python packages: pandas, numpy, requests, and logbook. These plugins are downloaded when you install the Norgate Data Python plugin using the pip command.
  4. Norgate Data Updater is installed and running.

Norgate Data Trial Subscription

Go to the trial subscription sign-up page, select “Python (Windows)” from the access methods, enter your email address, and agree to the privacy policy.

Select the packages you want to download, fill in your details, including name and phone number, and click the “Initiate trial” button at the bottom of the page. For the trial subscription, I suggest you select all the packages.

Installing Norgate Data Updater

Once you sign up for the trial subscription, you will receive an email with your username, password, and Norgate Data Updater download link.

Download and install the updater on your Windows machine. Open the application once you install it. Enter the username and password you received in the email. The updater will start downloading the historical data for your selected stock markets.  

You should see the following window once all the historical data is downloaded.

After downloading the historical data, Norgate Data repeatedly updates itself with real-time data from stock markets.

Once you install Norgate Data Updater, you can use the Norgate Data Python plugin to retrieve data in your Python application.

Installing norgatedata Python Plugin

The Norgate Data Python plugin is a Python package you can install via the following pip command.

pip install norgatedata --upgrade

You can update an existing installation via the following command:

pip install norgatedata --upgrade

Fetching Data Using Norgate Python Plugin

Before we delve deep into various Noragate Data endpoints, let’s see a basic example of how to retrieve Norgate Data in Python code.

We will import historical time series data for Microsoft.

The first step is to import the norgatedata module. Next, you can call the price_timeseries() method to retrieve historical time series data for a stock. You must pass the stock symbol to the symbol attribute of the price_timeseries() method.

import norgatedata
symbol = 'MSFT'
pricedata_recarray = norgatedata.price_timeseries(
    symbol = symbol
)
pricedata_recarray

By default, the price_timeseries() method retrieves all historical data for a company. You can pass the start date and end date to filter the data within specific dates. The interval attribute specifies the interval of the data, which can be daily (D), weekly (W), or monthly (M). The default is daily.

import norgatedata
 
symbol = 'MSFT'
start_date = '2022-12-01'
end_date = '2022-12-31'
interval = 'D'

pricedata_recarray = norgatedata.price_timeseries(
    symbol = symbol,
    start_date = start_date,
    end_date = end_date,
    interval = interval,
)

pricedata_recarray

I always recommend converting data into Pandas Dataframe for better viewing.

import pandas as pd

price_df = pd.DataFrame(pricedata_recarray)
price_df.head()

In the next section, you will see various Norgate Data Endpoint functions in action.

Norgate Data Endpoints

Norgate Data divides its functionalities into four endpoints:

  • Price and Volume Data
  • Time Series Data
  • Single Value Data
  • Security Lists

Price and Volume Data

The price and volume data endpoint returns historical OHLCV data for a stock. The endpoint contains price_timeseries() method, which you have already seen in action.

In this section, you will see some additional options available with the price_timeseries() method.

Data Formats

By default, the price_timeseries() method returns data in the Numpy recurrent array format. You can retrieve data in other formats such as Numpy n-dimensional array, and Pandas dataframe.

Pass pandas-dataframe as the parameter value to the timeseriesformat attribute of the price_timeseries() method to retrieve data in a Pandas dataframe.

symbol = 'MSFT'
start_date = '2022-12-01'
end_date = '2022-12-31'
interval = 'D'
timeseriesformat = 'pandas-dataframe'

pricedata_df = norgatedata.price_timeseries(
    symbol = symbol,
    start_date = start_date,
    end_date = end_date,
    interval = interval,
    timeseriesformat = timeseriesformat
)

pricedata_df.head()

Pass numpy-ndarray value for the timeseriesformat attribute to retrieve data in Numpy n-dimensional array format.

symbol = 'MSFT'
start_date = '2022-12-01'
end_date = '2022-12-31'
interval = 'D'
timeseriesformat = 'numpy-ndarray'

pricedata_ndarray = norgatedata.price_timeseries(
    symbol = symbol,
    start_date = start_date,
    end_date = end_date,
    interval = interval,
    timeseriesformat = timeseriesformat
)

pricedata_ndarray

Retrieving Weekly and Monthly Prices

Passing “W” or “M” as values for the interval attribute of the price_timeseries() method returns historical prices in weekly or monthly intervals, respectively.

symbol = 'MSFT'
start_date = '2022-01-01'
end_date = '2022-12-31'
interval = 'W' # for weekly
timeseriesformat = 'pandas-dataframe'

pricedata_df = norgatedata.price_timeseries(
    symbol = symbol,
    start_date = start_date,
    end_date = end_date,
    interval = interval,
    timeseriesformat = timeseriesformat
)

pricedata_df.head()
symbol = 'MSFT'
start_date = '2022-01-01'
end_date = '2022-12-31'
interval = 'M' # for monthly
timeseriesformat = 'pandas-dataframe'

pricedata_df = norgatedata.price_timeseries(
    symbol = symbol,
    start_date = start_date,
    end_date = end_date,
    interval = interval,
    timeseriesformat = timeseriesformat
)

pricedata_df.head()

Limiting Number of Retrieved Records

The price_timeseries() method returns all available historical stock prices by default. You can retrieve the N latest records using the limit attribute of the price_timeseries() method, where N is an integer.

symbol = 'MSFT'
start_date = '2022-01-01'
end_date = '2022-12-31'
interval = 'D' 
timeseriesformat = 'pandas-dataframe'
limit = 100

pricedata_df = norgatedata.price_timeseries(
    symbol = symbol,
    start_date = start_date,
    end_date = end_date,
    interval = interval,
    timeseriesformat = timeseriesformat,
    limit = limit
)

print(pricedata_df.shape)
pricedata_df.head()

Plotting Line Plot

You will often need to plot line plots using historical stock prices. To do so, I recommend retrieving data in Pandas dataframe format. You can then call the dataframe’s plot() method to plot a line plot for any dataframe column.

import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style("darkgrid")
sns.set_context("notebook")

pricedata_df['Close'].plot(figsize=(10,7))

Adjusted Price and Volume Data

Capital events and dividends affect stock price and volume data. You can make price and volume adjustment by passing norgatedata.StockPriceAdjustmentType.CAPITAL value to the priceadjust attribute of the price_timeseries() method. The output will show historical time series data with price volume adjustment.

symbol = 'MSFT'
start_date = '2022-12-01'
end_date = '2022-12-31'
interval = 'D' 
timeseriesformat = 'pandas-dataframe'

priceadjust = norgatedata.StockPriceAdjustmentType.CAPITAL

pricedata_df = norgatedata.price_timeseries(
    stock_price_adjustment_setting = priceadjust,
    symbol = symbol,
    start_date = start_date,
    end_date = end_date,
    interval = interval,
    timeseriesformat = timeseriesformat
)

pricedata_df.head()

Data Padding

By default, the price_timeseries() method doesn’t return stock prices for weekends or days when the stock market is closed. You can copy prior closing data for these days by passing norgatedata.PaddingType.ALLCALENDARDAYS value to the padding_setting attribute.  

For example, the output of the following script shows that the prior closing values for 2022-12-02 are copied for 2022-12-03 and 2022-12-04 since the later dates are weekends.

symbol = 'MSFT'
start_date = '2022-12-01'
end_date = '2022-12-31'
interval = 'D' 
timeseriesformat = 'pandas-dataframe'

priceadjust = norgatedata.StockPriceAdjustmentType.CAPITAL
padding_setting = norgatedata.PaddingType.ALLCALENDARDAYS

pricedata_df = norgatedata.price_timeseries(
    stock_price_adjustment_setting = priceadjust,
    padding_setting = padding_setting,
    symbol = symbol,
    start_date = start_date,
    end_date = end_date,
    interval = interval,
    timeseriesformat = timeseriesformat
)

pricedata_df.head()

Time Series Data

Norgate Data documentation has a dedicated time series data endpoint section. The price and volume endpoint also returns data in time series format, but it is not included in the Time series endpoint as per official documentation, which doesn’t make much sense to me.  

Index Constituents

The index_constituent_timeseries() method allows you to check if a company belongs to a particular index on specific dates. A return value of 1 indicates that the company belongs to a specific index, while a value of 0 shows that a company does not fall into a particular index.

The historical index constituent information is only available with Platinum or Diamond subscriptions.

You need to pass the symbol and index name to the symbol, and indexname attribute of the index_constituent_timeseries() method.

For example, the following script shows that Microsoft is a part of the S & P 100 index.

symbol = 'MSFT'
indexname = 'S&P 100'

idx = norgatedata.index_constituent_timeseries(
    symbol,
    indexname,
    timeseriesformat = "pandas-dataframe",
)

idx.head()

Similarly, Microsoft is not part of the “Nasdaq Biotechnology” index.

symbol = 'MSFT'
indexname = 'Nasdaq Biotechnology'

idx = norgatedata.index_constituent_timeseries(
    symbol,
    indexname,
    timeseriesformat = "pandas-dataframe",
)

idx.head()

You can check the official documentation for a list of all US, Australia, and Canada Historical Index Constituents.

You can also add the “Index Constituent” column to the records that the price_timeseries() method returns. To do so, you must retrieve data in a Pandas dataframe using the price_timeseries() method.

symbol = 'MSFT'
start_date = '2020-01-01'
end_date = '2021-12-31'
interval = 'D' 
timeseriesformat = 'pandas-dataframe'


pricedata_df = norgatedata.price_timeseries(

    symbol = symbol,
    start_date = start_date,
    end_date = end_date,
    interval = interval,
    timeseriesformat = timeseriesformat
)

pricedata_df.head()

Next, you can pass the Pandas dataframe with historical price and volume data to the pandas_dataframe attribute of the index_constituent_timeseries() method.

In the output, you will see the “Index Constituent” column added to the price and volume data.

symbol = 'MSFT'
indexname = 'Nasdaq Biotechnology'

idx = norgatedata.index_constituent_timeseries(
    symbol,
    indexname,
    pandas_dataframe = pricedata_df,
    timeseriesformat = "pandas-dataframe",
)

idx.head()

Major Exchange Listed

This indicator shows whether a stock was listed on a major exchange (such as NYSE, Nasdaq, NYSE American, NYSE Arca, Cboe BZX, IEX) or traded as an over the counter (OTC) /Pink Sheet stock on a specific date. It offers data on all Equities dating back to January 2000; no information is available before that time.

The major_exchange_listed_timeseries() method returns information about whether or not a stock is listed on a major exchange.

symbol =  "XOM"
majexch = norgatedata.major_exchange_listed_timeseries(
    symbol,
    timeseriesformat = "pandas-dataframe")

majexch.head()

Dividend Yield

This indicator calculates the sum of all split-adjusted ordinary dividends over the past 12 months and is calculated daily against the close price.

The dividend_yield_timeseries() method returns this information.

symbol =  "MSFT"
divyield = norgatedata.dividend_yield_timeseries(
    symbol,
    timeseriesformat = "pandas-dataframe")

divyield.tail()

Single Value Data

Single value data endpoint contains methods that retrieve single value data for securities, and futures meta for future markets, future markets sessions, and contracts.

Security Information

In this section, you will see methods that return single value information for securities.

Get Norgate Asset ID

You can access a security’s information using its asset id, which is an unchanging number. Asset id helps store security information in databases where the security name is likely to change.

The assetid() method returns the asset id of a security.

symbol =  "XOM"
asset_id = norgatedata.assetid(symbol)
print(asset_id)

Convert Asset ID to Symbol

Using the symbol() method, you can convert an asset ID back to a symbol name.

assetid = 139954
symbol = norgatedata.symbol(assetid)
print(symbol)

Domicile

The domicile() method returns the country name where the security operates. For instance, XOM operates in the USA.

symbol =  "XOM"
domicile = norgatedata.domicile(symbol)
print(domicile)

Similarly, “BUB.au” operates in Australia.

symbol =  "BUB.au"
domicile = norgatedata.domicile(symbol)
print(domicile

Trading Currency

The currency() method returns the currency in which a security operates.

symbol =  "XOM"
currency = norgatedata.currency(symbol)
print(currency)

Exchange Information

You can get the exchange name for a security using the exchange_name() method.

symbol =  "XOM"
exchange_sn = norgatedata.exchange_name(symbol)
print(exchange_sn)

The exchange_name_full() method returns the full name of the exchange.

symbol =  "XOM"
exchange_fn = norgatedata.exchange_name_full(symbol)
print(exchange_fn)

Security Full Name

The security_name() method returns the full name of a security.

symbol =  "XOM"
full_name = norgatedata.security_name(symbol)
print(full_name)

Base Type

The base_type() method returns a security’s base type, e.g., stock market, futures market, foreign exchange, etc.

symbol =  "XOM"
base_type = norgatedata.base_type(symbol)
print(base_type)

Financial and Business

You can get the financial summary for a stock using the financial_summary() method.

symbol =  "XOM"
fin_sum = norgatedata.financial_summary(symbol)
print(fin_sum)

The business_summary() method returns the business summary for a stock.

symbol =  "XOM"
bus_sum = norgatedata.business_summary(symbol)
print(bus_sum)

Futures Metadata

Futures metadata methods return information about future contracts, e.g., tick size, margins,  and agreements to buy or sell a specific asset at a predetermined price and date in the future.

Stock Tick Size

Stock tick size refers to a stock’s minimum price movement, typically measured in increments of one cent or less. Tick sizes are significant because they determine the smallest amount by which the price of a stock can change.

The tick_size() method returns the tick size value for a stock.

symbol='XOM'
tick_size = norgatedata.tick_size(symbol)
print(tick_size)

Stock Point Value

Stock point value refers to the dollar value of one point movement in the price of a stock. The point value of a stock can help determine the potential profit or loss of trade, as well as manage risk by setting stop-loss orders and determining position sizes.

The point_value() method returns the point value for a stock.

symbol='XOM'
point_value = norgatedata.point_value(symbol)
print(point_value)

Stock Margin

Margin refers to the amount of funds a trader must deposit with their broker to open a futures position.

The margin() method returns the future margin value for a stock.

symbol='XOM'
margin = norgatedata.margin(symbol)
print(margin)

Futures Market Symbols

Futures markets are financial markets where participants can trade standardized contracts to buy or sell an underlying asset at a specific price and date in the future.

The future_market_symbols() method returns symbols for all future markets.

market_symbols = norgatedata.futures_market_symbols()
print(market_symbols)

Get Market Name Using Symbols

The futures_market_name() method returns the name of a future market using the future market symbol as a parameter value.

market_symbol='GWM'
market_name = norgatedata.futures_market_name(market_symbol)
print(market_name)

Futures Market Session Symbols

A futures market session refers to the period during which trading in a futures market takes place. The future_market_session_symbols() method returns a list of all future market session symbols.

session_symbols = norgatedata.futures_market_session_symbols()
print(session_symbols)

Futures Market Session Contracts

You can get fundamental stock data, e.g., market cap, price-to-earnings ratio, current dividend yield, etc., using the fundamental() method.

session_symbol='FTDX'
session_contracts = norgatedata.futures_market_session_contracts(session_symbol)
print(session_contracts)

Fundamental Data

You can get fundamental stock data e.g, market cap, price to earnings ratio, current dividend yield, etc, using the fundamental() method.

Refer to the official documentation to see a list of all fields for fundamental data.

For example, the following code shows how to retrieve the latest stock market capitalization value.

symbol = 'MSFT'
fieldname = 'mktcap'

fundavalue,fundadate = norgatedata.fundamental(symbol, fieldname)
print(fundavalue)
print(fundadate)

Similarly, the following code shows how to retrieve outstanding shares for a stock.

symbol = 'MSFT'
fieldname = 'sharesoutstanding'

fundavalue,fundadate = norgatedata.fundamental(symbol, fieldname)
print(fundavalue)
print(fundadate)

Security Lists

The Norgate Data provides methods to retrieve security lists from various watchlists.  

Get All Available Watch Lists

The watchlists() method returns all available watchlist.

allwatchlists = norgatedata.watchlists()
print(allwatchlists)

Get Watchlists Symbols and Names

You can retrieve symbols and names of all stocks within a watchlist.

The watchlist_symbols() method returns a list of all stock symbols.

watchlistname = 'Nasdaq 100 + Q-50 Superset'

wl_symbols = norgatedata.watchlist_symbols(watchlistname)
print(wl_symbols)

The watchlist() method returns all stock names and ids within a watchlist.

watchlistname = 'Nasdaq 100 + Q-50 Superset'

wl_contents = norgatedata.watchlist(watchlistname)
print(wl_contents)

You can convert all stocks’ names and asset ids into a Pandas dataframe using the following script.

wl_contents_df = pd.DataFrame(wl_contents)
wl_contents_df.head()

Get Norgate Databases

You can get a list of all available Norgate Data databases using the database() method. The list of databases can give you an idea of the types of information you can retrieve from Norgate Data.

alldatabasenames = norgatedata.databases()
print(alldatabasenames)

The database_symbols() method returns a list of symbols for all entities within a database. For example, the following script returns a list of all US equities.

databasename = 'US Equities'
us_symbols = norgatedata.database_symbols(databasename)
print(us_symbols)

The following script returns a list of all AU equities.

databasename = 'AU Equities'
us_symbols = norgatedata.database_symbols(databasename)
print(us_symbols)

The database() method returns the contents of all entities within a database. For example, for US entities, the database() method returns the symbol, asset id, and security name of all stocks within the database.

databasename = 'US Equities'
us_contents = norgatedata.database(databasename)
us_contents_df = pd.DataFrame(us_contents)
us_contents_df.head()

With that, I will end the discussion on Norgate Data endpoints. You can look at the Norgate Data content tables to retrieve other information from Norgate Data. 

Norgate Data Plugins

In addition to Python, Norgate Data officially supports the following plugins:

  1. AmiBroker
  2. RightEdge
  3. Wealth-Lab
  4. Zipline (Windows)

Following are some third-party plugins available for Norgate Data.

  1. CandleScanner
  2. QuantumX
  3. RealTest
  4. Share Trade Tracker
  5. Stator
  6. System Trader
  7. TuringTrader
  8. XLQ2

Norgate Data Alternatives

Following are some alternatives to Norgate Data:

  1. Tiingo
  2. MetaStock
  3. Intrinio
  4. AlphaVantage

The Bottom Line

This Norgate Data Python Tutorial offers a valuable resource for algorithmic traders and quantitative analysts seeking to automate their trading activities using Norgate Data endpoints. The tutorial demonstrates how to call various endpoints using the Norgate Data Python plugin, allowing traders to integrate high-quality financial data into their Python applications. With Norgate Data’s reliable and accurate data, traders can make better-informed decisions and improve their overall trading performance. So, whether you are a seasoned trader or a beginner, follow the examples in this tutorial, and take the first step toward automating your trading activities.

Leave a Comment