How to Patch and Build Python 3.9.x for FIPS enabled Openssl

March 02, 2021

Introduction

In this post, we will see

  • Python 3.9.x patch for FIPS enabled Openssl
  • Test with Hashlib
  • Test with Cryptography module
  • Test with libcrypto shared library

In previous post, we saw how we built FIPS-enabled Openssl.

It is important to note that even you have FIPS enabled Openssl, still you need something to invoke this. Only setting to environment variable will not work if you are working from other language like Python. So, you need those methods in Python to get/set fips mode in Openssl.

Python 3.9.x patch

Note: This patch is tested for Python 3.9.0 and 3.9.2

This patch is built over https://bugs.python.org/issue27592 The older patch was exposing two methods FIPS_mode() and FIPS_mode_set() in Python.

Python 3.9 solves the issue when even your Openssl is fips enabled. Still, you are able to get md5. We solved this issue in Python 3.7.9 patch for FIPS

diff -aur Python-3.9.0__orig/Lib/ssl.py Python-3.9.0/Lib/ssl.py
--- Python-3.9.0__orig/Lib/ssl.py	2020-10-05 15:07:58.000000000 +0000
+++ Python-3.9.0/Lib/ssl.py	2021-03-02 04:23:32.026226000 +0000
@@ -111,6 +111,11 @@
     # LibreSSL does not provide RAND_egd
     pass
 
+try:
+    from _ssl import FIPS_mode, FIPS_mode_set
+except ImportError as e:
+    sys.stderr.write('error in importing\n')
+    sys.stderr.write(str(e))
 
 from _ssl import (
     HAS_SNI, HAS_ECDH, HAS_NPN, HAS_ALPN, HAS_SSLv2, HAS_SSLv3, HAS_TLSv1,
diff -aur Python-3.9.0__orig/Modules/Setup Python-3.9.0/Modules/Setup
--- Python-3.9.0__orig/Modules/Setup	2020-10-05 15:07:58.000000000 +0000
+++ Python-3.9.0/Modules/Setup	2021-03-02 04:24:28.071717000 +0000
@@ -207,14 +207,14 @@
 #_csv _csv.c
 
 # Socket module helper for socket(2)
-#_socket socketmodule.c
+_socket socketmodule.c
 
 # Socket module helper for SSL support; you must comment out the other
 # socket line above, and possibly edit the SSL variable:
-#SSL=/usr/local/ssl
-#_ssl _ssl.c \
-#	-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-#	-L$(SSL)/lib -lssl -lcrypto
+SSL=/usr/local/ssl
+_ssl _ssl.c \
+	-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
+	-L$(SSL)/lib -lssl -lcrypto
 
 # The crypt module is now disabled by default because it breaks builds
 # on many systems (where -lcrypt is needed), e.g. Linux (I believe).
diff -aur Python-3.9.0__orig/Modules/_ssl.c Python-3.9.0/Modules/_ssl.c
--- Python-3.9.0__orig/Modules/_ssl.c	2020-10-05 15:07:58.000000000 +0000
+++ Python-3.9.0/Modules/_ssl.c	2021-03-02 04:25:30.930669000 +0000
@@ -5394,6 +5394,20 @@
     return PyLong_FromLong(RAND_status());
 }
 
+static PyObject *
+_ssl_FIPS_mode_impl(PyObject *module) {
+    return PyLong_FromLong(FIPS_mode());
+}
+
+static PyObject *
+_ssl_FIPS_mode_set_impl(PyObject *module, int n) {
+    if (FIPS_mode_set(n) == 0) {
+        _setSSLError(ERR_error_string(ERR_get_error(), NULL) , 0, __FILE__, __LINE__);
+        return NULL;
+    }
+    Py_RETURN_NONE;
+}
+
 #ifndef OPENSSL_NO_EGD
 /* LCOV_EXCL_START */
 /*[clinic input]
@@ -5875,6 +5889,8 @@
     _SSL_ENUM_CRLS_METHODDEF
     _SSL_TXT2OBJ_METHODDEF
     _SSL_NID2OBJ_METHODDEF
+    _SSL_FIPS_MODE_METHODDEF
+    _SSL_FIPS_MODE_SET_METHODDEF
     {NULL,                  NULL}            /* Sentinel */
 };
 
diff -aur Python-3.9.0__orig/Modules/clinic/_ssl.c.h Python-3.9.0/Modules/clinic/_ssl.c.h
--- Python-3.9.0__orig/Modules/clinic/_ssl.c.h	2020-10-05 15:07:58.000000000 +0000
+++ Python-3.9.0/Modules/clinic/_ssl.c.h	2021-03-02 04:27:06.120295000 +0000
@@ -1204,6 +1204,45 @@
     return _ssl_RAND_status_impl(module);
 }
 
+PyDoc_STRVAR(_ssl_FIPS_mode__doc__,
+"FIPS Mode");
+
+#define _SSL_FIPS_MODE_METHODDEF    \
+    {"FIPS_mode", (PyCFunction)_ssl_FIPS_mode, METH_NOARGS, _ssl_FIPS_mode__doc__},
+
+static PyObject *
+_ssl_FIPS_mode_impl(PyObject *module);
+
+static PyObject *
+_ssl_FIPS_mode(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+    return _ssl_FIPS_mode_impl(module);
+}
+
+PyDoc_STRVAR(_ssl_FIPS_mode_set_doc__,
+"FIPS Mode Set");
+
+#define _SSL_FIPS_MODE_SET_METHODDEF    \
+    {"FIPS_mode_set", (PyCFunction)_ssl_FIPS_mode_set, METH_O, _ssl_FIPS_mode_set_doc__},
+
+static PyObject *
+_ssl_FIPS_mode_set_impl(PyObject *module, int n);
+
+static PyObject *
+_ssl_FIPS_mode_set(PyObject *module, PyObject *arg)
+{
+    PyObject *return_value = NULL;
+    int n;
+
+    if (!PyArg_Parse(arg, "i:FIPS_mode_set", &n)) {
+        goto exit;
+    }
+    return_value = _ssl_FIPS_mode_set_impl(module, n);
+
+exit:
+    return return_value;
+}
+
 #if !defined(OPENSSL_NO_EGD)
 
 PyDoc_STRVAR(_ssl_RAND_egd__doc__,

How to apply Patch

Download Python 3.9.2 source code from https://www.python.org/ftp/python/3.9.2/Python-3.9.2.tgz

wget https://www.python.org/ftp/python/3.9.2/Python-3.9.2.tgz
tar -xzf Python-3.9.2.tgz
cd Python-3.9.2

Assuming patch file patch.diff is placed outside Python folder.

patch -p1 < ../patch.diff

Compile Python

Now comes the tricker part. We need to carefully set the compiler flags and linker flags.

CFLAGS=-Wl,--enable-new-dtags,-rpath,/usr/local/ssl/lib LDFLAGS=-L/usr/local/ssl/lib CPPFLAGS=-I/usr/local/ssl/include LIBS=-lcrypto ./configure --enable-shared 

make

make install

If above steps goes well. Python is installed on your machine.

Test Python

export LD_LIBRARY_PATH=/usr/local/ssl/lib:/usr/local/lib

Run version command,

python3.9 --version
Python 3.9.2

Build Cryptography Module

Although, our work is complete with basic Python/hashlib. But, if somebody installed another python module cryptography, he/she can still able to calculate unsupported FIPS-encyption algorithm like md5.

Its a little complex trick, where we need to install cryptography module in a different way.

# upgrade pip
python3.9 -m pip install --upgrade pip

# install wheel
python3.9 -m pip install wheel

# Build cryptography wheel
CRYPTOGRAPHY_DONT_BUILD_RUST=1 CFLAGS="-I/usr/local/ssl/include" LDFLAGS="-L/usr/local/ssl/lib" python3.9 -m pip wheel --no-binary :all: cryptography==3.0

# This will create a wheel file in current directory with name: cryptography-3.0-cp39-cp39-linux_x86_64.whl

# install cryptography module now
CRYPTOGRAPHY_DONT_BUILD_RUST=1 CFLAGS="-I/usr/local/ssl/include" LDFLAGS="-L/usr/local/ssl/lib" python3.9 -m pip install cryptography-3.0-cp39-cp39-linux_x86_64.whl

Note: We are installing cryptography-3.0, not the latest one. As, it does not have support for our FIPS enabled Openssl (1.0.2t)

We have solved our most of the problem, now it the time to test things.

Testing - via Hashlib

import ssl

print(ssl.OPENSSL_VERSION)
print(ssl.FIPS_mode())

ssl.FIPS_mode_set(1)

print(ssl.FIPS_mode())

import hashlib
print(hashlib.sha1("test_str".encode('utf-8')).hexdigest())
print(hashlib.md5("test_str".encode('utf-8')).hexdigest())

Output

OpenSSL 1.0.2t-fips  10 Sep 2019
0
1
f9a90e7c1ff51236191623b84267d110c617118a
Traceback (most recent call last):
  File "/python_downloads/Python-3.9.2/t1.py", line 12, in <module>
    print(hashlib.md5("test_str".encode('utf-8')).hexdigest())
ValueError: [digital envelope routines: FIPS_DIGESTINIT] disabled for fips

Testing - via libcrypto

import sys
import ssl
import ctypes


libcrypto = ctypes.CDLL("libcrypto.so.1.0.0")

fips_mode = libcrypto.FIPS_mode
fips_mode.argtypes = []
fips_mode.restype = ctypes.c_int

fips_mode_set = libcrypto.FIPS_mode_set
fips_mode_set.argtypes = [ctypes.c_int]
fips_mode_set.restype = ctypes.c_int

text = b""

if __name__ == "__main__":
    print("OPENSSL_VERSION: {:s}".format(ssl.OPENSSL_VERSION))
    print("FIPS_mode(): {:d}".format(fips_mode()))
    print("FIPS_mode_set(1): {:d}".format(fips_mode_set(1)))
    print("FIPS_mode(): {:d}".format(fips_mode()))

    import hashlib
    print("SHA1: {:s}".format(hashlib.sha1(text).hexdigest()))
    print("MD5: {:s}".format(hashlib.md5(text).hexdigest()))

Output

OPENSSL_VERSION: OpenSSL 1.0.2t-fips  10 Sep 2019
FIPS_mode(): 0
FIPS_mode_set(1): 1
FIPS_mode(): 1
SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
Traceback (most recent call last):
  File "/python_downloads/Python-3.9.2/t1.py", line 26, in <module>
    print("MD5: {:s}".format(hashlib.md5(text).hexdigest()))
ValueError: [digital envelope routines: FIPS_DIGESTINIT] disabled for fips

Testing - via Cryptography Hazmat package

As I mentioned before, cryptography module is a dangerous one, as its package hazmat called Hazardous Materials provides access to lower level library where you can talk directly to openssl.

from cryptography.hazmat.backends.openssl.backend import backend
import cffi

ffi = cffi.FFI()
lib = backend._lib
print(lib.FIPS_mode())
lib.FIPS_mode_set(1)
print(lib.FIPS_mode())

from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.backends import default_backend
digest = hashes.Hash(hashes.MD5(), default_backend())
digest.update(b"abc")
digest.update(b"123")
print(digest.finalize())

Output

0
1
Traceback (most recent call last):
  File "/python_downloads/Python-3.9.2/t1.py", line 12, in <module>
    digest = hashes.Hash(hashes.MD5(), default_backend())
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/primitives/hashes.py", line 85, in __init__
    self._ctx = self._backend.create_hash_ctx(self.algorithm)
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 342, in create_hash_ctx
    return _HashContext(self, algorithm)
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/backends/openssl/hashes.py", line 36, in __init__
    self._backend.openssl_assert(res != 0)
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 234, in openssl_assert
    return binding._openssl_assert(self._lib, ok)
  File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 69, in _openssl_assert
    raise InternalError(
cryptography.exceptions.InternalError: Unknown OpenSSL error. This error is commonly encountered when another library is not cleaning up the OpenSSL error stack. If you are using cryptography with another library that uses OpenSSL try disabling it before reporting a bug. Otherwise please file an issue at https://github.com/pyca/cryptography/issues with information on how to reproduce this. ([_OpenSSLErrorWithText(code=101351587, lib=6, func=168, reason=163, reason_text=b'error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips')])

How to use Md5 if you really have to

Python3.9 has really started work towards FIPS. They have exposed a flag usedforsecurity in each hashing function via hashlib. Default value is True

Example

import ssl

print(ssl.OPENSSL_VERSION)
print(ssl.FIPS_mode())
ssl.FIPS_mode_set(1)
print(ssl.FIPS_mode())

import hashlib
print(hashlib.md5("test_str".encode('utf-8'), usedforsecurity=False).hexdigest())

You might question, I have enabled FIPS. Then why the hell, md5 still worked. Well, there are many situations where you would want to calculate md5, as its fast. And, many libraries where you might be checking for file changed or not, has support for only md5. So, you can not get away with md5.

The objective is that developer knows that he is using these algorithms and it is not for security.

Let me know if you have any query.

opensslfipssecuritypython

Similar Posts

Latest Posts