Serverless – Useful Resources ”; Previous Next The following resources contain additional information on Serverless. Please use them to get more in-depth knowledge on this. Useful Links on Serverless Serverless − Official Website of Serverless. Serverless Wiki − Wikipedia Reference for Serverless. Useful Books on Serverless To enlist your site on this page, please drop an email to [email protected] Print Page Previous Next Advertisements ”;
Discuss Serverless ”; Previous Next Serverless tutorial, as the name suggests, helps you deploy your lambda functions using the serverless functions. Lambda functions are becoming very popular because they relieve the user of the headache of maintaining a server, and also because they charge the user only for the amount of time and resources used. If your function runs for 500ms, you will be charged only for 500ms. Deploying lambda functions from your local machine to AWS has been simplified very significantly by the serverless framework, further relieving the user of the headache of creating the stack and the required resources. Serverless does everything for you to get your lambda functions running. We will primarily look at configuring the serverless framework in this tutorial. Two examples of popular serverless projects are provided at the end, for you to get an idea of the complete picture. Print Page Previous Next Advertisements ”;
Serverless – Include/Exclude ”; Previous Next We”ve already seen in the ”Deploying a function” chapter that to deploy functions from an existing project to AWS lambda, you need to modify the functions to take event and context as arguments and you need to add a serverless.yml file in the project folder, with the functions defined.Then hitting serverless deploy does the job. Quite frequently, especially if you are asked to migrate some functions from an extensive existing project to AWS Lambda, you are faced with a size challenge.If your project is large enough, you may very well exceed the size limits imposed by AWS on Lambda functions (250 MB, including the application code and its dependencies). And some dependencies like NumPy occupy a lot of space themselves. For instance, NumPy is ~80 MB large, SciPy is also in the same ballpark, and so on.In such scenarios, you have very limited space left for the application code and require a method of excluding unnecessary files from the lambda deployment package. Luckily, serverless makes this quite easy. Include and exclude fields As you would have guessed, you can specify files and folders that you want to exclude from the deployment build using the ”exclude” tag. All the files/ folders not specified in the exclude section are included by default. Then what is the use of the ”include” tag? Well, if you want a folder to be excluded in general, but want to include just a few files or sub-folders within that folder, you can specify those files/sub-folders within the ”include” tag. This way, all other files within that folder will be excluded, and only those specified within the ”include” section will remain. The example below will explain this better. service: influx-archive-pipeline provider: name: aws runtime: python3.6 stage: prod region: us-east-2 profile: yash-sanghvi timeout: 900 memorySize: 1024 # you can add packaging information here package: include: – src/models/config.py – src/models/lambda_apis/** – src/models/scheduled_lambdas/** exclude: – docs/** – models/** – notebooks/** – references/** – reports/** – src/data/** – src/visualization/** – src/models/** – utils/** functions: user_details_api: handler: src/models/lambda_apis/user_details_api.sync_user_details events: – http: path: details/{user_id} method: get integration: lambda cors: true monitoring_lambda: handler: src/models/scheduled_lambdas/monitoring_lambda.periodic_monitoring events: – schedule: cron(15 16 * * ? *) As you can from the above severless.yml file, we are excluding most of the folders within the root folder that contains the serverless.yml. We have even excluded the src/models folder. However,we want to include 2 sub-folders and 1 file within src/models. Therefore, those have been specifically added in the ”include” section. Please note that any file/ folder that is not a part of the exclude section will be included by default. Note the path of the two lambda functions. They both lie in src/models. While src/models is excluded by default, these functions specifically lie in sub-folders that are mentioned in the include section. Therefore they will execute without any issues. If I added a function lying in, say, src/data, that would not be allowed, because all the contents of src/data have been excluded. Please note that specifying /** indicates that all content (files/sub-folders) within that folder gets covered. Thus if the docs folder contains 10 sub-folders and 12 files, all of which need to be excluded, then -docs/** does the job. We don”t need to mention each file/folder separately. Print Page Previous Next Advertisements ”;
Serverless – Layer Creation ”; Previous Next What are layers? Layers are a way of isolating code blocks. Say you want to import the NumPy library in your application. You trust the library and there”s hardly any chance that you will be making changes in the source code of that library. Therefore, you won”t like it if the source code of NumPy clutters your application workspace. Speaking very crudely, you would simply want NumPy to sit somewhere else, isolated from your application code. Layers allow you to do exactly that. You can simply bundle all your dependencies (NumPy, Pandas, SciPy, etc.) in a separate layer, and then simply reference that layer in your lambda function within serverless. And boom! All the libraries bundled within that layer can now be imported into your application. At the same time, your application workspace remains completely uncluttered. You simply see the application code to edit. Photo by Iva Rajovic on Unsplash, indicative of code separation in layers The really cool thing about layers is that they can be shared across functions. Say you deployed a lambda function with a python-requirements layer that contains NumPy and Pandas. Now, if another lambda function requires NumPy, you need not deploy a separate layer for this function. You can simply use the layer of the previous function and it will work well with the new function as well. This will save you a lot of precious time during deployment. After all, you will be deploying only the application code. The dependencies are already present in an existing layer. Therefore,several developers keep the dependencies layer in a separate stack. They then use this layer in all other applications. This way, they don”t need to deploy the dependencies again and again. After all, the dependencies are quite heavy. NumPy library itself is approx. 80 MB large. Deploying dependencies every time you make changes to your application code (which may measure just a few KBs) will be quite inconvenient. And adding a dependencies layer is just one example. There are several other use-cases. For example, the example given on serverless.com concerns the creation of GIFs using the FFmpeg tool. In that example, they have stored the FFmpeg tool in a layer. In all, AWS Lambda allows us to add a maximum of 5 layers per function. The only condition is that the total size of the 5 layers and the application should be less than 250 MB. Creating python-requirements layer Now let”s see how the layer containing all the dependencies can be created and deployed using serverless. To do that, we need the serverless-python-requirements plugin.This plugin only works with Serverless 1.34 and above. So you may want to upgrade your Serverless version if you have a version <1.34. You can install the plugin using − sls plugin install -n serverless-python-requirements Next, you add this plugin in the plugins section of your serverless.yml, and mention it”s configurations in the custom section − plugins: – serverless-python-requirements custom: pythonRequirements: dockerizePip: true layer: true Over here, dockerizePip − true enables the usage of docker and allows you to package all the dependencies in a docker container. We”ve discussed about packaging using docker in the previous chapter. layer − true tells serverless that the python requirements should be stored in a separate layer. Now, at this point, you may be wondering that how does serverless understand which dependencies to package? The answer, as mentioned in the plugins chapter, lies in the requirements.txt file. Once the layer plugin and custom configurations have been defined, you can add the layer to your individual functions within serverless as follows − functions: hello: handler: handler.hello layers: – { Ref: PythonRequirementsLambdaLayer } The keyword PythonRequirementsLambdaLayer comes from the CloudFormation Reference.In general, it is derived from the layer”s name. The syntax is ”LayerNameLambdaLayer” (TitleCased, without spaces). In our case, since the layer name is python requirements, the reference becomes PythonRequirementsLambdaLayer. If you aren”t sure about the name of your lambda layer, you can get it in the following steps − Run sls package Open .serverless/cloudformation-template-update-stack.json Search for ”LambdaLayer” Using an existing layer from another function in the same region Like I mentioned in the beginning, a really cool thing about layers is the ability to use existing layers in your function. This can be done easily by using the ARN of your existing layer.The syntax to add an existing layer to a function using the ARN is quite straightforward − functions: hello: handler: handler.hello layers: – arn:aws:lambda:region:XXXXXX:layer:LayerName:Y That”s it. Now the layer with the specified ARN will work with your function. If the layer contains the NumPy library, you can simply go ahead and call import numpy in your ”hello” function. It will run without any error. If you are wondering from where you can get the ARN, it is quite simple actually. Simply navigate to the function containing the layer in the AWS Console, and click on ”Layers”. Of course, if the layer doesn”t belong to your account, then it either needs to be publicly shared or shared specifically with your account. More on that later. Also, keep in mind that the layer should be compatible with your application. Don”t expect a layer compatible with node.js runtime to run with a function created in python3.6 runtime. Non-requirements/ generic layers As mentioned in the beginning, the layers serve the main function of isolating your code blocks. Therefore, they don”t need to contain just dependencies. They can contain any piece of code that you specify. Calling layer: true within pythonRequirements within custom is a kind of a shortcut made possible by the serverless-python-requirements plugin. However, to create a generic layer, the syntax within serverless.yml, as explained in the serverless docs, is as follows − layers: hello: path: layer-dir # required,
Serverless – Quick Guide ”; Previous Next Serverless – Introduction What is Serverless? Well, the name gives you quite some hints. Computation without the headache of maintaining a server − that”s the crux of serverless computing (or serverless in short). The concept is quite revolutionary and disruptive. It has been widely adopted. Several fresh applications begin by designing a serverless backend, and the legacy applications with dedicated servers are also slowly migrating to the serverless architecture. So what has led to the widespread adoption of serverless? As with everything, the economics have made serverless quite favorable. You see, with serverless, you only pay for what you use. Imagine you need to perform some routine maintenance on your database every day. This process may take about 10 minutes every day. Now, in the absence of serverless computing, your maintenance cron may be residing in a server. Unless you have something else to do with your server in the remaining time, you may end up paying for 24 hours, for a task that takes 10 minutes. Quite a waste of money, right? What if you were told that there is a new service that will charge you exactly for those 10 minutes that your maintenance cron takes to execute? Won”t you want to simply switch to that new service? That”s exactly why serverless adoption has been so swift and widespread. It has brought down the backend bills for several organizations, and also reduced their server maintenance headache. The cloud service provider (AWS, Azure, etc.) takes the headache of making sure that the serverless application is available exactly when required, and in the quantity required. Thus, during high loads, you may invoke multiple serverless applications, while you may invoke a single application during normal load. And of course,you will pay for the extra invocations only for the duration of the high load. What is Serverless, again? The concept explained above seems great, but how do you implement it? You need a framework for that. It is called, uh, serverless. The serverless framework helps us develop and deploy functions/ applications designed to run in a serverless fashion. The framework goes a step ahead and takes care of the deployment of the entire stack required for our serverless functions to run. What is a stack? Well, the stack comprises of all the resources that you will require for deploying, storing, and monitoring your serverless applications. It includes the actual function/ application, storage containers, monitoring solutions, and a lot more.For example, in the context of AWS, your stack will consist of your actual Lambda function, S3 bucket for your function files, Cloudwatch resources linked to your function, and so on.The serverless framework creates this entire stack for us. This allows us to focus completely on our function. Serverless takes away the headache of maintaining a server and serverless (framework) takes away the headache of creating and deploying the stack necessary to run our function. The serverless framework also takes care of assigning the necessary permissions to our functions/ applications.Some applications (examples of which we will see in this tutorial) even require databases to be linked to them. Serverless framework again takes care of creating and linking the DBs. How does serverless know what to include in the stack and which permissions to provide? All of it is mentioned in the serverless.yml file, which will be our main focus in this tutorial. More on it in the upcoming chapters. Serverless in AWS There are many services of AWS that fall under the umbrella of ”serverless computing”. You can find the entire organized list here. There are Compute services, Integration services, even Data Storage services (yes, AWS even has Serverless DBs). We will be focusing on the AWS Lambda functions throughout the tutorial. So what is AWS Lambda? The AWS Lambda website defines it as follows − AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes. In layperson terms, AWS Lambda is your window to serverless computing on AWS. It is AWS Lambda that has made the serverless concept so popular. All you need to do is define your function and the trigger to your function, and the function will be invoked exactly when you want it to be invoked, and you will be charged only for the time the function takes to execute. What”s more,you can link AWS Lambda with almost every other service that AWS provides − EC2, S3, dynamoDB, and so on. Therefore, if you have already been a part of the AWS ecosystem, then Lambda integration is quite seamless. If you are new to the AWS ecosystem like I was when I first learned about AWS Lambda, it will act as a good gateway to the AWS universe. In this tutorial, we will be learning all about the deployment of AWS Lambda functions using the serverless framework. Are you excited? Then move on to the next chapter to get started. Serverless – Installing Serverless installation has already been covered in another tutorialspoint tutorial. Reproducing it here, with some modifications and additions. Step 1 − Install nodejs To begin with, you need to first install nodejs. You can check whether nodejs is installed in your machine or not by opening the Command Prompt and typing node -v. If it is installed, you will get the version number of node. Otherwise, you can download and install node from here. Step 2 − Install serverless using the npm command You can install serverless using the following command (npm stands for node package manager) − npm install -g serverless You can check whether it got successfully installed by running serverless create –help. If serverless is successfully installed, you should see the help screen for the create
Serverless – Scheduled Lambdas ”; Previous Next Often, you require your function to be invoked at fixed intervals. It can be once a day, twice a week, once a minute during weekdays, and so on.Serverless provides two types of events to invoke functions at fixed frequencies. They are cron events and rate events. Cron Event The cron event has much more flexibility than the rate event. The only downside is that it is not as straightforward to understand as the rate event. The syntax of the cron expression is defined in the AWS documentation − cron(minutes hours day-of-month month day-of-week year) As can be seen, the cron expression consists of 6 fields. Each field can take in some accepted values, and some, as AWS calls them, wildcards. Let”s have a look at the accepted values first − minutes − 0-59 hours − 0-23 day-of-month − 1-31 month − 1-12 or JAN-DEC day-of-week − 1-7 or SUN-SAT year − 1970-2199 Now that the accepted values are clear, let us have a look at the wildcards. There are a total of 8 wildcards possible in the cron expression (some allowed for all 6 fields and some only for specific fields). Listing them down here − * (asterisk, allowed for all 6 fields) − This is the most popular wildcard. It simply says that all values of the field are included. A * in the hours field would mean that the cron would run every hour. A * in the day-of-month field would mean that the cron will run every day. , (comma, allowed for all 6 fields) − This is used to specify more than one value. For example. If you want your cron to run at the 5th, 7th, and 9th minute of every hour, your minute field would look like 5,7,9.Similarly, MON,TUE,WED,THU,FRI in the day-of-week field could mean that the cron should run only on weekdays. – (dash, allowed for all 6 fields) − This wildcard specifies ranges. In the previous wildcard example, in order to specify weekdays, instead of specifying 5 comma-separated-values, we could have simply written MON-FRI ? (question mark, only allowed for day-of-month and day-of-week) − This is like a don”t-care wildcard. If you have specified MON in the day-of-week field, you don”t care what date Monday falls on. Therefore, you will enter ? in place of day-of-month. Similarly, if you want the cron to run on the 5th of every month, you will enter 5 in the day-of-month field and ? in the day-of-week field, because you don”t care what the day is on the 5th of every month. Note that the AWS documentation clearly states that you cannot use * for both day-of-week and day-of-month fields. If you use * for one, you have to use ? for the other / (forward slash, allowed for the 5 fields except for day-of-month) − This field specifies increments. If you enter 0/2 in the hours field, this cron will run every even hour (0, 0+2, 0+2+2, and so on). If you specify 1/2 in the hours field, this cron will run every odd hour (1, 1+2, 1+2+2, and so on). As you would have guessed, the value preceding the / is the starting value and the value succeeding it defines the increment. L (only allowed for day-of-month and day-of-week) − Specifies the last day of the month or the last day of the week W (only allowed for day-of-month) − This specifies a weekday (Monday to Friday) nearest to that specific day of the month. So if you specify 8W in the day-of-month field, and it corresponds to a weekday, say Tuesday, then the cron will fire on 8th itself. But if 8 corresponds to a weekend day, say Saturday, then the cron will be triggered on 7th (Friday). If 8th corresponds to Sunday, then the cron will fire on 9th (Monday). This is one of the least used wildcards. # (only allowed for day-of-week) − This is a very special wildcard, best understood by an example. Say you want a cron to run on Mother”s day. Now, Mother”s day falls on 2nd Sunday of May every year.Therefore, your month field would contain MAY or 5. But how do you specify 2nd Sunday? Come in the hashtag. The expression is 0#2. The value preceding the wildcard is the day of the week (0 for Sunday,1 for Monday, and so on). The value succeeding the wildcard specifies the occurrence. Therefore, 2 here refers to the 2nd occurrence of Sunday or the second Sunday. Now, to define a cron trigger for your lambda function, all you need to do is specify the cron expression within the events key in the function in serverless.yml. functions: cron_triggered_lambda: handler: handler.hello events: – schedule: cron(10 03 * * ? *) #run at 03:10 (UTC) every day. Some Examples Given below are some examples of cron expressions − cron(30 15 ? * MON-FRI *) − Triggered at 15:30 (UTC) on every weekday cron(0 9 ? 6 0#3 *) − Triggered at 09:00 (UTC) on the third Sunday of June (Father”s Day) cron(0/15 * ? * MON *) − Triggered every 15 minutes on Mondays cron(0/30 9-18 ? * MON-FRI *) − Triggered every 30 minutes from 9 AM to 5:30 PM on weekdays (corresponding to office hours at several places) Rate Event This is much more straightforward compared to cron expressions. The syntax is simply rate(value unit). For example, rate(5 minutes). The value can be any positive integer and the allowed units are minute(s), hour(s), day(s). Defining a rate trigger for your lambda function is similar to defining a cron trigger. functions: rate_triggered_lambda: handler: handler.hello events: – schedule: rate(10 minutes) #run every 10 minutes
Serverless – Regions, Memory-size, Timeouts ”; Previous Next We saw how to deploy our first function using serverless in the previous chapter. In this chapter, we will look at some configurations that we can perform on the function. We will primarily look at the region, the memory-size, and the timeout. Region By default, all lambda functions deployed using serverless are created in the us-east-1 region. If you want your lambda functions to get created in a different region, you can specify that in the provider. provider: name: aws runtime: python3.6 region: us-east-2 profile: yash-sanghvi It is not possible to specify different regions for different functions within the same serverless.yml file. You should include only the functions belonging to a single region in a particular serverless.yml file. Function belonging to a separate region can be deployed using a separate serverless.yml file. MemorySize AWS Lambda allocates CPU in proportion to the memory chosen. With the recently announced changes, you can choose up to 10GB of RAM for your lambda function (it was ~3 GB earlier). The higher the RAM chosen, the higher is the CPU allocated, the faster your function executes, the lower is the execution time. AWS Lambda charges you for the GB-s consumed. Therefore, if a function on 1 GB RAM takes 10 seconds to execute and it takes 5 seconds to execute on a 2 GB RAM, you will be charged the same amount for both the invocations. Whether the time halves on doubling the memory depends a lot on the nature of your function, and you may or may not benefit by increasing the memory. The key takeaway is that the amount of memory allotted is an important setting for each lambda function and one you would like to have control of. With serverless, it is quite easy to set the default value of the memory-size for the functions defined within your serverless.yml file. It is also possible to define different memory-sizes for different functions. Let us see how. Setting default memory-size for all the functions The default values are always mentioned in the provider. This value will be inherited by all the functions within that serverless.yml. The memorySize key is used for setting this value.The value is expressed in MB. provider: name: aws runtime: python3.6 region: us-east-2 profile: yash-sanghvi memorySize: 512 #will be inherited by all functions In case you don”t specify the memorySize in the provider, nor in individual functions, the default value of 1024 will be considered. Setting custom memory-size for some functions In case you want some functions to have a value different than the default memory, then you can specify it in the functions section of serverless.yml. functions: custom_memory_func: #will override the default memorySize handler: handler.custom_memory memorySize: 2048 default_memory_func: #will inherit the default memorySize from provider handler: handler.default_memory Timeout Just like memorySize, the default value of the timeout (in seconds) can be set in the provider, and custom timeouts for individual functions can be specified in the functions section. If you don”t specify either the global or custom timeouts, the default value is 6 seconds. provider: name: aws runtime: python3.6 region: us-east-2 profile: yash-sanghvi memorySize: 512 #will be inherited by all functions timeout: 50 #will be inherited by all functions functions: custom_timeout: #will override the default timeout handler: handler.custom_memory timeout: 30 default_timeout_func: #will inherit the default timeout from provider handler: handler.default_memory Make sure to keep the timeout at a conservative value. It should not be so small that your function times out very frequently, nor should it be so large that a bug in your function leaves you with an outrageous bill to pay. Print Page Previous Next Advertisements ”;
Serverless – API Gateway Triggered Lambdas ”; Previous Next API Gateways are another popular method of triggering lambdas, just like cron/rate events. Basically, you get a URL endpoint for your lambda function. This URL belongs to the API Gateway connected to your lambda. Whenever you invoke the URL,either in the browser or through an application, your lambda function gets invoked. In this chapter, we will see how to connect an API Gateway to your lambda function using the serverless framework, and how to test it. HTTP Events To link an API Gateway to a lambda function, we need to create HTTP events in the function definition in serverless.yml. The following example shows how to link your lambda function(s) to a REST API and trigger it using the GET request. functions: user_details_api: handler: handler.send_user_details events: – http: path: details/{user_id} method: get integration: lambda-proxy cors: true location_api: handler: handler.send_location events: – http: path: location/{user_id} method: get integration: lambda-proxy cors: true Let”s unpack the keys one by one. We will only discuss the first function from the above list (user_details_api). The concepts covered below hold for the other function as well. The value of the path specifies the address after the endpoint for invoking the URL. Both the functions defined in the above example will share the same endpoint, but one will be invoked using endpoint/details/{user_id}, while the other will be invoked by endpoint/location/{user_id} The elements within curly brackets are path parameters. I can send any value in place of user_id and the lambda function can be programmed to return details for that particular user (see the example function below). The value of the method indicates the request method. The popular methods are get and post. There are several other methods as well. Diving into the details of these methods is outside the scope of this chapter. There is another post on tutorialspoint which you can refer to for the details. The integration field specifies how the lambda function will be integrated with the API Gateway. The default is lambda-proxy, while the other options which are possible are lambda, http, http-proxy, mock. The most widely used options out of these two are lambda and lambda-proxy. In layperson terms, lambda-proxy gives the entire control to your lambda function, while lambda gives some control to the API Gateway and some to the lambda function. If you choose lambda-proxy as the integration type, then the entire HTTP request will be passed in the raw form to your lambda function, and the response sent by the lambda function is passed without alteration to the client which made the request. Therefore,you have to define the statusCode and headers in the response of your lambda function. If you choose lambda as the integration type, your API Gateway can make changes to the received request,before passing it on to the lambda function. Similarly, it can also modify the response sent by the lambda function before forwarding it to the client. The API Gateway adds the status code and the headers to the response, and therefore, the lambda function just has to worry about sending the body. Both options have their advantages and disadvantages. You can use lambda-proxy if you prefer simplicity. You can choose lambda if you are okay with some complexity (since you will have to worry about the lambda function”s code as well as the API Gateway”s configuration), but require more control. You can read more about the difference between these two types here.Out of the other integration types, http and http-proxy are used when integrating the API Gateway with an HTTP backend instead of the lambda function, so irrelevant for us. mockis used when you just want to test the API without invoking the backend. The cors − true configuration enables CORS (Cross-Origin Resource Sharing). In layperson terms, this means that you allow requests from servers from another domain. Without cors − true, only requests from the same domain will be allowed. Of course, instead of allowing all domains, you can allow only some specific domains. To understand how to do that, see the documentation. There are a lot more configurations possible within serverless, for API-Gateway triggered lambdas. You are strongly recommended to go through the documentation, or at least bookmark the link so that you can look it up whenever required. Sample Lambda Function At this point, you may be wondering that you created the API Gateway triggered function, but how do you access the path parameters in the lambda function? The following sample lambda function in python will answer that. We basically use the ”pathParameters” property, when the integration type is lambda-proxy. import json def lambda_handler(event, context): # TODO implement # print(event) #helps you see the entire input request. The printed output can be found in CloudWatch logs user = event[”pathParameters”][”user_id”] return { ”statusCode”: 200, ”body”: json.dumps(”Hello ” + str(user)) } Accessing the Endpoint Now, another question that you may be having is how do you access the endpoint. There are multiple ways of doing that. The first way is through serverless deployment. Whenever you deploy a function, or multiple functions through a service, the endpoint is shown at the end of serverless deploy. The second method is through the Lambda console. If you navigate to your function on the lambda console, you can see the API Gateway attached to it. Clicking on it should reveal the endpoint. Please note that, as discussed above, all the functions within a service share the same endpoint.The path property differentiates the actual trigger URL of one function from another. References Serverless API Gateway documentation lambda-proxy vs lambda integration Print Page Previous Next Advertisements ”;
Serverless – Packaging Dependencies ”; Previous Next In the previous chapter, we saw how to use plugins with serverless. We specifically looked at the Python Requirements plugin and saw how it can be used to bundle dependencies like numpy, scipy, pandas, etc.with your lambda function”s application code. We even saw an example of deploying a function requiring the numpy dependency. We saw that it ran well locally but on the AWS Lambda console, you”d have encountered an error if you are on a Windows or Mac machine. Let”s understand why the function runs locally but doesn”t run after deployment. If you look at the error message, you get some hints. I”m specifically referring to one line − ”Importing the numpy C-extensions failed.” Now, many important python packages like numpy, pandas, scipy, etc.require the compilation of C-extensions. If we compile them on a Windows or a Mac machine, then Lambda (linux environment) will throw up an error when trying to load them. So the important question is, what can be done to avoid this error. Come in docker! What is docker? According to Wikipedia, docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. If you scan the Wikipedia page of docker a bit more, you will come across some more relevant statements − Docker can package an application and its dependencies in a virtual container that can run on any Linux, Windows, or macOS computer. This enables the application to run in a variety of locations, such as on-premises, in a public cloud, and/or in a private cloud. I think it should be very clear after the above statements. We have an error coming up because C-extensions compiled on Windows/Mac don”t work in Linux. We can simply bypass that error by packaging the application in a container that can run on any OS. What docker does in the background to achieve this OS-level virtualization is beyond the scope of this chapter. Installing docker You can head over to https://docs.docker.com/engine/install/ for the installation of Docker Desktop. If you are using Windows 10 Home Edition,the Windows version should be at least 1903 (May 2019 update). Therefore, you may want to upgrade your Windows 10 OS before installing Docker Desktop. No such limitations apply to Windows Pro or Enterprise versions. Using dockerizePip in serverless Once Docker Desktop has been installed on your machine, you need to make only the following addition to your serverless.yml file to package your applications and dependencies using docker − custom: pythonRequirements: dockerizePip: true Please note, that if you have been following along with me since the previous chapter, it is likely that you have already deployed code to lambda once.This would have created a static cache in your local storage. By default, serverless would use that cache to bundle dependencies, and therefore, docker container won”t be created.Therefore, to force serverless to create use docker, we will add another statement to pythonRequirements − custom: pythonRequirements: dockerizePip: true useStaticCache: false #not necessary if you will be deploying the code to lambda for the first time. This last statement is not necessary if you are deploying to lambda for the first time. In general, you should set useStaticCache to true, since that will save you some packaging time when you haven”t made any changes to the dependencies or the way they have to be packaged. With these additions, the serverless.yml file now looks like − service: hello-world-python provider: name: aws runtime: python3.6 profile: yash-sanghvi region: ap-south-1 functions: hello_world: handler: handler.hello timeout: 6 memorySize: 128 plugins: – serverless-python-requirements custom: pythonRequirements: dockerizePip: true useStaticCache: false #not necessary if you will be deploying the code to lambda for the first time. Now, when you run the sls deploy -v command, make sure that docker is running in the background. On Windows, you can just search for Docker Desktop in the Start menu and double click on the app. You will soon get a message that it is running. You can also verify this through the small popup near the battery icon in Windows. If you can see the docker icon there, it is running. Now when you run your function on the AWS Lambda console, it would work. Congratulations!! However, in your ”Function Code” section on the AWS Lambda console, you would be seeing a message saying ”The deployment package of your Lambda function “hello-world-python-dev-hello_world” is too large to enable inline code editing. However, you can still invoke your function.” Seems like the addition of the Numpy dependency has made the bundle size too large and as a result, we cannot even edit our application code in the lambda console. How do we solve that problem? Head on to the next chapter to find out. References Docker Desktop Serverless Python Requirements Print Page Previous Next Advertisements ”;