Jupyter Notebook - Running under IIS on Windows

I use Jupyter Notebooks quite a lot for personal development, and to do this I set up an instance on a DigitalOcean droplet. However I also wanted to do something similar for some tasks at work which can access resources only available in our internal network. Since we use Windows that means I need to do something a little different. I could just use Anaconda to install and load it, but I find it a little clunky, I don't like how it tries to manage everything for you, I wanted a slightly nicer url than https://localhost:8000 and more importantly I wanted it to just be there without having to start anything up. So I set out to get Jupyter Notebook running on IIS using the ASP.NET Core Module V2 on http://notebook.local.

Make sure Python, IIS and the ASP.NET Core Hosting Bundle are installed. Open up a Terminal (I'm using bash, Powershell is probably file but I don't like it), create an new directory where our notebook application will be served, setup a venv there, and run the activation script, upgrade pip so that it doesn't complain at us and then finally install Jupyter Notebook:

$ cd ~/source/repos
$ mkdir notebook
$ python -m venv venv
$ source venv/Scripts/activate
$ pip install --upgrade pip
$ pip install notebook
To start with let's just run it on its own to check that it works:

$ jupyter notebook
[I 20:39:42.322 NotebookApp] The port 8888 is already in use, trying another port.
[I 20:39:42.328 NotebookApp] Serving notebooks from local directory: C:\Users\sean\source\external
[I 20:39:42.329 NotebookApp] Jupyter Notebook 6.1.4 is running at:
[I 20:39:42.329 NotebookApp] http://localhost:8889/?token=e27aa8d7754e1bb078dfdf48e7c55032ac3551360389fd65
[I 20:39:42.329 NotebookApp]  or http://127.0.0.1:8889/?token=e27aa8d7754e1bb078dfdf48e7c55032ac3551360389fd65
[I 20:39:42.329 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 20:39:42.476 NotebookApp]

    To access the notebook, open this file in a browser:
        file:///C:/Users/sean/AppData/Roaming/jupyter/runtime/nbserver-260-open.html
    Or copy and paste one of these URLs:
        http://localhost:8889/?token=e27aa8d7754e1bb078dfdf48e7c55032ac3551360389fd65
     or http://127.0.0.1:8889/?token=e27aa8d7754e1bb078dfdf48e7c55032ac3551360389fd65

So here you can see why I'm doing all this - http://localhost:8888 isn't quite as nice as https://notebook.local. Next let's create a logs folder where our stdout logging will live and notebooks folder where we'll keep our notebook files.

$ mkdir logs notebooks

Next we'll create a Python script called launch.py that will launch Jupyter:

import os, sys
from IPython.terminal.ipapp import launch_new_instance
from IPython.lib import passwd
import warnings

# set the password you want to use
notebook_password = "test"

sys.argv.append("notebook")
sys.argv.append("--IPKernelApp.pylab='inline'")
sys.argv.append("--NotebookApp.ip='*'")
sys.argv.append("--NotebookApp.port=" + os.environ["ASPNETCORE_PORT"])
sys.argv.append("--NotebookApp.open_browser=False")
sys.argv.append("--NotebookApp.notebook_dir=./notebooks")
sys.argv.append("--NotebookApp.password=" + passwd(notebook_password))

launch_new_instance()

This script is kind of interesting, but we'll go into it later once we've got everything up and running. Next we will need to create a Web.config file that'll be used by IIS and ANCMV2 to call this script with the Python executable and libraries from our venv. So let's create the following:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <handlers>
      <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified" />
    </handlers>
    <aspNetCore processPath=".\venv\Scripts\python.exe" 
                arguments="launch.py"
                stdoutLogEnabled="true" 
                stdoutLogFile="logs\stdout">
      <environmentVariables>
        <environmentVariable name="PYTHONPATH" value="." />
      </environmentVariables>    
    </aspNetCore>
  </system.webServer>
</configuration>

Next we'll set up the new Site in IIS - so open up IIS Manager (hit Start and start typing "Internet Information Services" - it'll appear in the suggestions) then:

  1. in the left-hand menu right click the top-level node, hit "Add Website..."
  2. in the dialog that appears call the site whatever you like
  3. in the "Physical Path" section click the "..." button, and choose the directory your notebooks live in
  4. in the "Binding" section type "notebook.local" in the "Host name" section

Now nearly everything's in place we just need to modify our local DNS to resolve "notebook.local" to 127.0.0.1 so add the following line to C:/Windows/System32/drivers/etc/hosts (you'll need to be an Administrator to do so):

127.0.0.1    notebook.local

So now we have a Website bound to "notebook.local" running under IIS, which is configured to use ASP.NET Core Runtime Module V2 to launch a Python script that will run Jupyter Notebook. Let's fire up a browser and navigate to http://notebook.local and enter the password test:

It loads nicely so let's just create a simple Python 3 notebook to confirm everything works end-to-end:

Excellent. Ok so let's rewind a little bit and ask the question: why do we use a special script instead of having our Web.config file launch jupyter notebook directly? Well that would be ideal, but a bit of a hack required. Notice in launch.py that we grab the ASPNETCORE_PORT environment variable and use that as our port? Well that contains the port IIS wants to communicate with our app in - so we need to hook Jupyter up to listen on that port. While it is possible to call jupyter notebook --port 8080 to specify the port we want to use, it's actually not possible to use the ASPNETCORE_PORT variable in our Web.config. These environment variables are not expanded in our arguments="..." attribute, (see issue #117 in the aspnet/AspNetCoreModule repo on GitHub) - so we need to run a script that grabs this from the Environment and sets it up. It's a bit hacky, but it gets us where we need to be.

IIS - Annoying error with .NET Core

I deployed a .NET Core application recently into an environment which previously had only run .NET Framework apps, but got a tricky HTTP Error 500.19 - Internal Server Error suggesting there was a problem with my Web.config:


What was also peculiar was that when I tried to enable stdout logging using the IIS UI I got an error "There was an error while performing this operation" with blank "Details" and "Error" but with the "Filename" set to the path to my Web.config. This was true for any of the configuration sections:


However after a lot of head scratching and googling turned up nothing my colleague realised that this environment was missing the Windows Server Hosting bundle, available from the .NET Core downloads section under the "Other Windows Downloads" section. Once this was installed everything worked as expected.