Python - How to Maintain Quality Build Process Using Pylint and Unittest Coverage With Minimum Threshold Values
Introduction It is very important to introduce few process so that your code and…
August 19, 2020
In previous post (Trigger Email on Blob Trigger), we saw how we can create such automation pipeline to get email notification every time someone changes our storage container.
But, that was completely manual.
Lets see how we can create all this with full automation with the help of Azure Arm templates.
Like previous post, this post assumes you have storage containers already with you. This automation does not create any storage account and containers.
Complete code and template files are placed at our Github repo for Azure Template{:target=“_blank”}
This has two parts:
azure_arm_template
file_monitor
Lets take a look at Azure Template file{:target=“_blank”}
This template file creates following resources:
Azure Function App
(type: Microsoft.Web/sites)
With required configuration like Sendgrid API key
Server Farm
(type: Microsoft.Web/serverfarms)
Actual Function
(type: Microsoft.Web/sites/functions)
This defines the actual bindings from storage container to sendgrid
Misc
Configuration and Bindings
Lets have the complete file here:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"sites_file_monitor_function_app_name": {
"defaultValue": "file-monitor-function-app",
"type": "String"
},
"siteName": {
"type": "string",
"defaultValue": "[concat('site', uniqueString(resourceGroup().id))]"
},
"storageAccountName": {
"type": "string",
"defaultValue": "<StorageContainerName>"
},
"storageAccountId": {
"type": "string",
"defaultValue": "<ResourceIdForStorageContainer>"
}
},
"variables": {
"hostingPlanName": "[concat(parameters('siteName'),'-plan')]",
"errorText": "error",
"appSettingsHelp": "app setting help"
},
"resources": [
{
"type": "Microsoft.Web/sites",
"apiVersion": "2018-11-01",
"name": "[parameters('sites_file_monitor_function_app_name')]",
"location": "[resourceGroup().location]",
"kind": "functionapp",
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', variables('hostingPlanName'))]"
],
"properties": {
"enabled": true,
"siteConfig": {
"appSettings": [
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=', parameters('storageAccountName'), ';AccountKey=', listKeys(parameters('storageAccountId'),'2015-05-01-preview').key1)]"
},
{
"name": "FUNCTIONS_WORKER_RUNTIME",
"value": "node"
},
{
"name": "WEBSITE_NODE_DEFAULT_VERSION",
"value": "10.14.1"
},
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~1"
},
{
"name": "FILE_MONITOR_SENDGRID_APIKEY",
"value": "<SENDGRID_API_KEY>"
}
]
},
"hostNameSslStates": [
{
"name": "[concat(parameters('sites_file_monitor_function_app_name'), '.azurewebsites.net')]",
"sslState": "Disabled",
"hostType": "Standard"
},
{
"name": "[concat(parameters('sites_file_monitor_function_app_name'), '.scm.azurewebsites.net')]",
"sslState": "Disabled",
"hostType": "Repository"
}
],
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('hostingPlanName'))]",
"reserved": false,
"isXenon": false,
"hyperV": false,
"scmSiteAlsoStopped": false,
"clientAffinityEnabled": true,
"clientCertEnabled": false,
"hostNamesDisabled": false,
"containerSize": 1536,
"dailyMemoryTimeQuota": 0,
"httpsOnly": false,
"redundancyMode": "None"
}
},
{
"type": "Microsoft.Web/serverfarms",
"apiVersion": "2018-02-01",
"name": "[variables('hostingPlanName')]",
"location": "[resourceGroup().location]",
"properties":{
"name":"[variables('hostingPlanName')]",
"computeMode":"Dynamic"
},
"sku":{
"name":"Y1",
"tier":"Dynamic",
"size":"Y1",
"family":"Y",
"capacity":0
}
},
{
"type": "Microsoft.Web/sites/config",
"apiVersion": "2018-11-01",
"name": "[concat(parameters('sites_file_monitor_function_app_name'), '/web')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', parameters('sites_file_monitor_function_app_name'))]"
],
"properties": {
"numberOfWorkers": -1,
"defaultDocuments": [
"Default.htm",
"Default.html",
"Default.asp",
"index.htm",
"index.html",
"iisstart.htm",
"default.aspx",
"index.php"
],
"netFrameworkVersion": "v4.0",
"phpVersion": "5.6",
"requestTracingEnabled": false,
"remoteDebuggingEnabled": false,
"httpLoggingEnabled": false,
"logsDirectorySizeLimit": 35,
"detailedErrorLoggingEnabled": false,
"publishingUsername": "$file-monitor-function-app",
"scmType": "None",
"use32BitWorkerProcess": true,
"webSocketsEnabled": false,
"alwaysOn": false,
"managedPipelineMode": "Integrated",
"virtualApplications": [
{
"virtualPath": "/",
"physicalPath": "site\\wwwroot",
"preloadEnabled": false
}
],
"loadBalancing": "LeastRequests",
"experiments": {
"rampUpRules": []
},
"autoHealEnabled": false,
"cors": {
"allowedOrigins": [
"https://functions.azure.com",
"https://functions-staging.azure.com",
"https://functions-next.azure.com"
],
"supportCredentials": false
},
"localMySqlEnabled": false,
"ipSecurityRestrictions": [
{
"ipAddress": "Any",
"action": "Allow",
"priority": 1,
"name": "Allow all",
"description": "Allow all access"
}
],
"scmIpSecurityRestrictions": [
{
"ipAddress": "Any",
"action": "Allow",
"priority": 1,
"name": "Allow all",
"description": "Allow all access"
}
],
"scmIpSecurityRestrictionsUseMain": false,
"http20Enabled": false,
"minTlsVersion": "1.2",
"ftpsState": "AllAllowed",
"reservedInstanceCount": 0
}
},
{
"type": "Microsoft.Web/sites/functions",
"apiVersion": "2018-11-01",
"name": "[concat(parameters('sites_file_monitor_function_app_name'), '/my-file-monitor')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', parameters('sites_file_monitor_function_app_name'))]"
],
"properties": {
"config": {
"bindings": [
{
"type": "blobTrigger",
"direction": "Trigger",
"name": "myBlob",
"connection":"AzureWebJobsStorage",
"path": "gtest/{name}"
},
{
"type": "sendGrid",
"direction": "out",
"extension": {
"id": "Microsoft.Azure.WebJobs.Extensions.SendGrid", "version": "3.0.0"
},
"name": "$return",
"apiKey": "FILE_MONITOR_SENDGRID_APIKEY"
}
]
},
"test_data": "samples-workitems/workitem.txt"
}
},
{
"type": "Microsoft.Web/sites/hostNameBindings",
"apiVersion": "2018-11-01",
"name": "[concat(parameters('sites_file_monitor_function_app_name'), '/', parameters('sites_file_monitor_function_app_name'), '.azurewebsites.net')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', parameters('sites_file_monitor_function_app_name'))]"
],
"properties": {
"siteName": "file-monitor-function-app-ProfilesPRDLocB",
"hostNameType": "Verified"
}
}
]
}
StorageContainerName
, and use your desired container nameResourceIdForStorageContainer
and use your resource id.
To get it,
Properties
. Click on itStorage account resource ID
./subscriptions/<your subscription id>/resourceGroups/<resource group>/providers/Microsoft.Storage/storageAccounts/<storage account name>
SENDGRID_API_KEY
to have your actual api key for SendgridI will use command line tool: az
for deployment purpose. You need to install this.
To install this, refer to: Azure CLI Download{:target=“_blank”}
Lets follow the steps:
1. az login
Run az login
It will take you to browser, ask you to login.
Once login, close your browser tab. On your console, you will see similar message:
You have logged in. Now let us find all the subscriptions to which you have access...
[
{
"cloudName": "AzureCloud",
"homeTenantId": "<some id>",
"id": "<some id>",
"isDefault": true,
"managedByTenants": [],
"name": "<your subscription name>",
"state": "Enabled",
"tenantId": "<some id>",
"user": {
"name": "<your email>",
"type": "user"
}
},
{
"cloudName": "AzureCloud",
"homeTenantId": "<some id>",
"id": "<some id>",
"isDefault": false,
"managedByTenants": [],
"name": "<your subscription name>",
"state": "Enabled",
"tenantId": "<some id>",
"user": {
"name": "<your email>",
"type": "user"
}
},
]
If you have more than one Azure subscription, it will show all of them here. Only one of them will be set by default.
See flag: isDefault
If you want to switch to other azure subscription, use following command:
az account set --subscription "your subscription name"
2. Run deployment command
az deployment group create --resource-group "<resource group name>" --template-file azure_arm_template/template.json
If successful, it should show you a success message and list of all the resources it created.
This part has the code for Azure function which performs two things:
See the code at github folder
Its a node.js project. There are major two files here:
index.js
It has the main code which receives the event and just sends an email.
function.js
This has the two bindings. One for storage container trigger, and other for Sendgrid.
First go inside folder
Run following command:
zip -r monitor.zip *
Run az login
if you haven’t done before.
Run following command:
az functionapp deployment source config-zip -g "<resource group name>" -n "Function App Name" --src monitor.zip
If everything succeeds, it will show a success message.
This code will help you to deploy your resources with no manual efforts. There are some configuration changes which you need to do in above code or template files. Which you can easily bring out in form of parameters.
I hope it helps.
Introduction It is very important to introduce few process so that your code and…
Lets implement a command shell by using a command dispatcher. The objective is…
Introduction Lets assume we have a csv something similar to following: Python…
Introduction This post is about hosting ElasticSearch cluster on dockerised…
Problem Statement I have a drupal module, where there is a file of extension…
Introduction In previous posts, we saw how to build FIPS enabled Openssl, and…
Introduction This post has the complete code to send email through smtp server…
Introduction In a normal email sending code from python, I’m getting following…
Introduction In one of my app, I was using to talk to . I have used some event…
Introduction So you have a Django project, and want to run it using docker image…
Introduction It is very important to introduce few process so that your code and…
Introduction In this post, we will see a sample Jenkin Pipeline Groovy script…