How to Patch and Build Python 3.7.9 for FIPS enabled Openssl

February 26, 2021

Introduction

In this post, we will see

  • Python 3.7.9 patch for FIPS enabled Openssl
  • Test with Hashlib
  • Test with Cryptography module
  • Test with libcrypto shared library

In previous post, we saw how we built FIPS-enabled Openssl.

It is important to note that even you have FIPS enabled Openssl, still you need something to invoke this. Only setting to environment variable will not work if you are working from other language like Python. So, you need those methods in Python to get/set fips mode in Openssl.

Python 3.7.9 patch

This patch is built over https://bugs.python.org/issue27592 The older patch was exposing two methods FIPS_mode() and FIPS_mode_set() in Python. However, if you try md5 with hashlib, it still worked. Why?

In python’s SMART hashlib, it tries to take help from Openssl first, if Openssl refuses, it has its own implementations of these hashing algorithms.

My patch also restricts that, if Openssl refuses for some algorithm. Hashlib will also fail.

diff -aur Python-3.7.9__orig/Lib/hashlib.py Python-3.7.9/Lib/hashlib.py
--- Python-3.7.9__orig/Lib/hashlib.py	2020-08-15 05:20:16.000000000 +0000
+++ Python-3.7.9/Lib/hashlib.py	2021-02-26 07:25:23.295243000 +0000
@@ -124,7 +124,11 @@
         f()
         # Use the C function directly (very fast)
         return f
-    except (AttributeError, ValueError):
+    except AttributeError:
+        return __get_builtin_constructor(name)
+    except ValueError as ve:
+        if str(ve) == '[digital envelope routines: FIPS_DIGESTINIT] disabled for fips':
+            raise ValueError('unsupported hash type ' + name)
         return __get_builtin_constructor(name)
 
 
diff -aur Python-3.7.9__orig/Lib/ssl.py Python-3.7.9/Lib/ssl.py
--- Python-3.7.9__orig/Lib/ssl.py	2020-08-15 05:20:16.000000000 +0000
+++ Python-3.7.9/Lib/ssl.py	2021-02-25 12:16:28.834165000 +0000
@@ -111,6 +111,11 @@
     # LibreSSL does not provide RAND_egd
     pass
 
+try:
+    from _ssl import FIPS_mode, FIPS_mode_set
+except ImportError as e:
+    sys.stderr.write('error in importing\n')
+    sys.stderr.write(str(e))
 
 from _ssl import (
     HAS_SNI, HAS_ECDH, HAS_NPN, HAS_ALPN, HAS_SSLv2, HAS_SSLv3, HAS_TLSv1,
diff -aur Python-3.7.9__orig/Modules/Setup.dist Python-3.7.9/Modules/Setup.dist
--- Python-3.7.9__orig/Modules/Setup.dist	2020-08-15 05:20:16.000000000 +0000
+++ Python-3.7.9/Modules/Setup.dist	2021-02-25 12:16:47.802632000 +0000
@@ -204,14 +204,14 @@
 #_csv _csv.c
 
 # Socket module helper for socket(2)
-#_socket socketmodule.c
+_socket socketmodule.c
 
 # Socket module helper for SSL support; you must comment out the other
 # socket line above, and possibly edit the SSL variable:
-#SSL=/usr/local/ssl
-#_ssl _ssl.c \
-#	-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-#	-L$(SSL)/lib -lssl -lcrypto
+SSL=/usr/local/ssl
+_ssl _ssl.c \
+	-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
+	-L$(SSL)/lib -lssl -lcrypto
 
 # The crypt module is now disabled by default because it breaks builds
 # on many systems (where -lcrypt is needed), e.g. Linux (I believe).
diff -aur Python-3.7.9__orig/Modules/_ssl.c Python-3.7.9/Modules/_ssl.c
--- Python-3.7.9__orig/Modules/_ssl.c	2020-08-15 05:20:16.000000000 +0000
+++ Python-3.7.9/Modules/_ssl.c	2021-02-25 12:17:47.908325000 +0000
@@ -5262,6 +5262,20 @@
     return PyLong_FromLong(RAND_status());
 }
 
+static PyObject *
+_ssl_FIPS_mode_impl(PyObject *module) {
+    return PyLong_FromLong(FIPS_mode());
+}
+
+static PyObject *
+_ssl_FIPS_mode_set_impl(PyObject *module, int n) {
+    if (FIPS_mode_set(n) == 0) {
+        _setSSLError(ERR_error_string(ERR_get_error(), NULL) , 0, __FILE__, __LINE__);
+        return NULL;
+    }
+    Py_RETURN_NONE;
+}
+
 #ifndef OPENSSL_NO_EGD
 /* LCOV_EXCL_START */
 /*[clinic input]
@@ -5743,6 +5757,8 @@
     _SSL_ENUM_CRLS_METHODDEF
     _SSL_TXT2OBJ_METHODDEF
     _SSL_NID2OBJ_METHODDEF
+    _SSL_FIPS_MODE_METHODDEF
+    _SSL_FIPS_MODE_SET_METHODDEF
     {NULL,                  NULL}            /* Sentinel */
 };
 
diff -aur Python-3.7.9__orig/Modules/clinic/_ssl.c.h Python-3.7.9/Modules/clinic/_ssl.c.h
--- Python-3.7.9__orig/Modules/clinic/_ssl.c.h	2020-08-15 05:20:16.000000000 +0000
+++ Python-3.7.9/Modules/clinic/_ssl.c.h	2021-02-25 12:18:51.207415000 +0000
@@ -972,6 +972,46 @@
     return _ssl_RAND_status_impl(module);
 }
 
+PyDoc_STRVAR(_ssl_FIPS_mode__doc__,
+"FIPS Mode");
+
+#define _SSL_FIPS_MODE_METHODDEF    \
+    {"FIPS_mode", (PyCFunction)_ssl_FIPS_mode, METH_NOARGS, _ssl_FIPS_mode__doc__},
+
+static PyObject *
+_ssl_FIPS_mode_impl(PyObject *module);
+
+static PyObject *
+_ssl_FIPS_mode(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+    return _ssl_FIPS_mode_impl(module);
+}
+
+PyDoc_STRVAR(_ssl_FIPS_mode_set_doc__,
+"FIPS Mode Set");
+
+#define _SSL_FIPS_MODE_SET_METHODDEF    \
+    {"FIPS_mode_set", (PyCFunction)_ssl_FIPS_mode_set, METH_O, _ssl_FIPS_mode_set_doc__},
+
+static PyObject *
+_ssl_FIPS_mode_set_impl(PyObject *module, int n);
+
+static PyObject *
+_ssl_FIPS_mode_set(PyObject *module, PyObject *arg)
+{
+    PyObject *return_value = NULL;
+    int n;
+
+    if (!PyArg_Parse(arg, "i:FIPS_mode_set", &n)) {
+        goto exit;
+    }
+    return_value = _ssl_FIPS_mode_set_impl(module, n);
+
+exit:
+    return return_value;
+}
+
+
 #if !defined(OPENSSL_NO_EGD)
 
 PyDoc_STRVAR(_ssl_RAND_egd__doc__,

How to apply Patch

Download Python 3.7.9 source code from https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tgz

wget https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tgz
tar -xzf Python-3.7.9.tgz
cd Python-3.7.9

Assuming patch file patch.diff is placed outside Python folder.

patch -p1 < ../patch.diff

Compile Python

Now comes the tricker part. We need to carefully set the compiler flags and linker flags.

CFLAGS=-Wl,--enable-new-dtags,-rpath,/usr/local/ssl/lib LDFLAGS=-L/usr/local/ssl/lib CPPFLAGS=-I/usr/local/ssl/include LIBS=-lcrypto ./configure --enable-shared 

make

make install

If above steps goes well. Python is installed on your machine.

Test Python

If you simply try running python, you will get an error:

python3.7 --version
python3.7: error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory

You need to set LD_LIBRARY_PATH

export LD_LIBRARY_PATH=/usr/local/ssl/lib:/usr/local/lib

Now if you run version command,

python3.7 --version
Python 3.7.9

It works.

Build Cryptography Module

Although, our work is complete with basic Python/hashlib. But, if somebody installed another python module cryptography, he/she can still able to calculate unsupported FIPS-encyption algorithm like md5.

Its a little complex trick, where we need to install cryptography module in a different way.

# upgrade pip
python3.7 -m pip install --upgrade pip

# install wheel
python3.7 -m pip install wheel

# Build cryptography wheel
CRYPTOGRAPHY_DONT_BUILD_RUST=1 CFLAGS="-I/usr/local/ssl/include" LDFLAGS="-L/usr/local/ssl/lib" python3.7 -m pip wheel --no-binary :all: cryptography==3.0

# This will create a wheel file in current directory with name: cryptography-3.0-cp37-cp37m-linux_x86_64.whl

# install cryptography module now
CRYPTOGRAPHY_DONT_BUILD_RUST=1 CFLAGS="-I/usr/local/ssl/include" LDFLAGS="-L/usr/local/ssl/lib" python3.7 -m pip install cryptography-3.0-cp37-cp37m-linux_x86_64.whl

Note: We are installing cryptography-3.0, not the latest one. As, it does not have support for our FIPS enabled Openssl (1.0.2t)

We have solved our most of the problem, now it the time to test things.

Testing - via Hashlib

import ssl

print(ssl.OPENSSL_VERSION)
print(ssl.FIPS_mode())

ssl.FIPS_mode_set(1)

print(ssl.FIPS_mode())

import hashlib
print(hashlib.sha1("test_str".encode('utf-8')).hexdigest())
print(hashlib.md5("test_str".encode('utf-8')).hexdigest())

Output

OpenSSL 1.0.2t-fips  10 Sep 2019
0
1
ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/hashlib.py", line 124, in __get_openssl_constructor
    f()
ValueError: [digital envelope routines: FIPS_DIGESTINIT] disabled for fips

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/hashlib.py", line 250, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/lib/python3.7/hashlib.py", line 131, in __get_openssl_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
f9a90e7c1ff51236191623b84267d110c617118a
Traceback (most recent call last):
  File "/python_downloads/test1.py", line 12, in <module>
    print(hashlib.md5("test_str".encode('utf-8')).hexdigest())
AttributeError: module 'hashlib' has no attribute 'md5'

This logging is done by hashlib. See the line in between: [digital envelope routines: FIPS_DIGESTINIT] disabled for fips

Testing - via libcrypto

import sys
import ssl
import ctypes


libcrypto = ctypes.CDLL("libcrypto.so.1.0.0")

fips_mode = libcrypto.FIPS_mode
fips_mode.argtypes = []
fips_mode.restype = ctypes.c_int

fips_mode_set = libcrypto.FIPS_mode_set
fips_mode_set.argtypes = [ctypes.c_int]
fips_mode_set.restype = ctypes.c_int

text = b""

if __name__ == "__main__":
    print("OPENSSL_VERSION: {:s}".format(ssl.OPENSSL_VERSION))
    print("FIPS_mode(): {:d}".format(fips_mode()))
    print("FIPS_mode_set(1): {:d}".format(fips_mode_set(1)))
    print("FIPS_mode(): {:d}".format(fips_mode()))

    import hashlib
    print("SHA1: {:s}".format(hashlib.sha1(text).hexdigest()))
    print("MD5: {:s}".format(hashlib.md5(text).hexdigest()))

Output

OPENSSL_VERSION: OpenSSL 1.0.2t-fips  10 Sep 2019
FIPS_mode(): 0
FIPS_mode_set(1): 1
FIPS_mode(): 1
ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/hashlib.py", line 124, in __get_openssl_constructor
    f()
ValueError: [digital envelope routines: FIPS_DIGESTINIT] disabled for fips

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/hashlib.py", line 250, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/lib/python3.7/hashlib.py", line 131, in __get_openssl_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
Traceback (most recent call last):
  File "/python_downloads/test1.py", line 26, in <module>
    print("MD5: {:s}".format(hashlib.md5(text).hexdigest()))
AttributeError: module 'hashlib' has no attribute 'md5'

Testing - via Cryptography Hazmat package

As I mentioned before, cryptography module is a dangerous one, as its package hazmat called Hazardous Materials provides access to lower level library where you can talk directly to openssl.

from cryptography.hazmat.backends.openssl.backend import backend
import cffi

ffi = cffi.FFI()
lib = backend._lib
print(lib.FIPS_mode())
lib.FIPS_mode_set(1)
print(lib.FIPS_mode())

from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.backends import default_backend
digest = hashes.Hash(hashes.MD5(), default_backend())
digest.update(b"abc")
digest.update(b"123")
print(digest.finalize())

Output

0
1
Traceback (most recent call last):
  File "/python_downloads/test1.py", line 12, in <module>
    digest = hashes.Hash(hashes.MD5(), default_backend())
  File "/usr/local/lib/python3.7/site-packages/cryptography/hazmat/primitives/hashes.py", line 85, in __init__
    self._ctx = self._backend.create_hash_ctx(self.algorithm)
  File "/usr/local/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 342, in create_hash_ctx
    return _HashContext(self, algorithm)
  File "/usr/local/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/hashes.py", line 36, in __init__
    self._backend.openssl_assert(res != 0)
  File "/usr/local/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 234, in openssl_assert
    return binding._openssl_assert(self._lib, ok)
  File "/usr/local/lib/python3.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 77, in _openssl_assert
    errors_with_text,
cryptography.exceptions.InternalError: Unknown OpenSSL error. This error is commonly encountered when another library is not cleaning up the OpenSSL error stack. If you are using cryptography with another library that uses OpenSSL try disabling it before reporting a bug. Otherwise please file an issue at https://github.com/pyca/cryptography/issues with information on how to reproduce this. ([_OpenSSLErrorWithText(code=101351587, lib=6, func=168, reason=163, reason_text=b'error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips')])

Side-Effects and Future work

As a precaution, you must note that md5 will completely be blocked with this process. We can use other implementation for md5, if we really want so. Python3.9 has starting support for FIPS, but is under development. It is dependent on Openssl-3.0 which is under development. (https://wiki.openssl.org/index.php/OpenSSL_3.0)

Future version of Python 3

I have tried changes for Python 3.9 as well.

Let me know if you have any query.

opensslfipssecuritypython

Similar Posts

Latest Posts