Setting and Using Jupyter Notebook Environment Variables

Ready to level up your Jupyter Notebook skills? In this tutorial, you’ll not only learn about the power of environment variables, but you’ll also discover the best practices for using them in your notebook. You’ll be a pro at managing your settings and configurations in no time!

Before we dive in, you can skip ahead using the table of contents and follow along using the Jupyter notebook.

What are Environment Variables?

Environment variables are values you pass to a computer’s operating system or an application at runtime. You use these values to configure the operating system’s behavior or applications like Jupyter notebook. Typical use cases are storing settings like file paths, user preferences, and other settings.

The printenv command lists all of your environment variables. You’ll learn about this command in a minute. You can easily see the path of my home directory, my name, and that I’m running Ubuntu.

What Is Jupyter Notebook?

Jupyter Notebook is an open-source web application that allows users to create and share documents that contain live code, equations, visualizations, and narrative text. 

It’s a powerful tool for data analysis and scientific computing. I use it for data exploration and when prototyping trading strategies.

Environment Variables Python Tutorial

Before we open up Jupyter notebook, let’s learn how to print out the current variables using the terminal, as I demonstrated above.

List Operating System Environment Variables

You can list your existing environment variables using printenv in a Unix-like operating system or the set command in windows.

Now that we’ve covered the basics let’s learn how to work with environment variables in Jupyter Notebook.

There are typically two ways to set an environment variable using Python:

1. Set it directly.
2. Read from a file.

Setting and Getting Environment Variables

To set an environment variable in Jupyter Notebook, you can use Python to set it using the os.environ or use magic commands. Let’s cover the os.environ function first.

Using OS

This os.environ function sets environment variables on the operating system where the Jupyter Notebook server is running.

To set the environment variable MY_VAR to the value my_value, enter the following:

import os
os.environ['MY_VAR'] = 'my_value'

You can assess the values of environment variables using the os.environ class and environment variable names. You don’t have to import the os module again if you have already imported it.

my_var = os.environ['MY_VAR']
my_var # Output : "my_value"

The os.environ class returns a Key error exception if you want to access a non-existing environment variable. 

my_var = os.environ['MY_VAR2']
my_var # the "MY_VAR2" variable does not exist

Alternatively, you can access environment variables using the os.getenv function. Let’s grab the MY_VAR variable we set earlier. 

my_var = os.getenv('MY_VAR')
my_var # Output : "my_value"

The os.getenv() function doesn’t return an exception when you try to access a non-existent environment variable. Instead, it returns None.

my_var = os.getenv('MY_VAR2')
print(my_var) # Output : None

Behind the scenes, the os.getenv() method returns a dictionary that can return a default value if an environment variable does not exist.

my_var = os.getenv('MY_VAR2', 'default value')
print(my_var) # Output: "default value"

Using Magic Commands

Now let’s cover magic commands to access and set Jupyter Notebook environment variables in Jupyter notebook. Magic commands are special commands built into Jupyter Notebook prefixed with %.

%set_env variable_name=value syntax. The %env magic command will list all your environment variables.

%env MY_MAGIC_VAR=my_value
%set_env MY_OTHER_MAGIC_VAR=my_other_value
%env

You can also print the value of a single environment using the %env command followed by the environment variable name.

%env MY_MAGIC_VAR # Output: "my_value"

Finally, you can use expansions to assign the value of a Python variable to an environment variable.

env_var = "new_env_var"
%env MY_MAGIC_VAR2= $env_var
# Output "new_env_var"

Unsetting Environment Variables

You can unset or delete an environment variable in Python using the del command with the os.environ class.

del os.environ['MY_MAGIC_VAR']

Reading Dotenv Environment Variables

Python-dotenv is a package that enables you to load environment variables directly from a .env file. This is useful for storing sensitive information like database credentials or API keys. It’s also helpful for keeping application configurations such as dev or production environments.

The .env file is a simple text file containing key-value pairs where the keys are the variable names. You can have as many variables as you would like. 

To use python-dotenv, install the package using pip. Remember to prepend an exclamation point if running the code in Jupyter notebook.

pip install python-dotenv # for command line
!pip install python-dotenv # for Jupyter notebook

After installing, use the load_dotenv()method from the dotenv module to load .env file key-value pairs into your environment.

from dotenv import load_dotenv
load_dotenv()
print(os.environ.get('MY_OTHER_VAR'))

# Output : my_other_value

Notice how we now have MY_OTHER_VALUE, as it wasn’t available to us.

Using Environment Variables for Sensitive Information

One of the most common use cases is to store sensitive information like API keys or database credentials. I use .env files for almost all the code I share for obvious reasons. This allows me to separate sensitive data and ensures I don’t commit it to shared source code.

Top Five Tips Using Environment Variables with Jupyter Notebook

1. Use descriptive and unique names for your environment variables to avoid conflicts.
2. Remember that environment variables are case-sensitive. Use the XCU specification of all uppercase with underscores if you want to do it right.
3. Consider storing sensitive information in environment variables to keep them separate from your code.
4. Use the printenv or set command from a terminal or prepending a ! when within Jupyter notebook to list environment variables.
5. Use the python-dotenv package to manage environment variables in a .env file.

The Bottom Line

This article explains how to access and set environment variables in Jupyter Notebook using the Python os module and magic commands. You also learned how to automatically load environment variables from .env files using the python-dotenv module. 

These valuable insights benefit aspiring Python programmers and budding data scientists, equipping them with the tools to manage environment variables from local systems and encrypted files seamlessly.

Now is the perfect time to unleash the potential of your Jupyter notebooks by incorporating environment variables – take action and embark on your data-driven journey today!

Frequently Asked Questions