Table of content
1.API first development - Why, how and what
2.API design: an example
3.API first development with OpenAPI/Swagger
4.Hosting your visualisations
5.Integrating with Postman
6.OpenAPI Generator: generate API compliant code
7.Integrating with your build process: maven or gradle plugin
8.Serverless on AWS: OpenAPI, API Gateway Lambda and SAM
9.Springfox
10.Conclusion
Intro
- I’ll start of with a lecture about API first development and it’s advantages.(Jump to section)
- We will discover how to visualize our API specs. (Jump to section)
- Generate a postman collection from your OpenAPI definition. (Jump to section)
- Next I’ll show you how to generate code that is completely compliant with your API specs. (Jump to section)
- We then dive into how to integrate this in your build process by using maven or gradle plugins. (Jump to section)
- To finish it up I’ll demonstrate how to use it in a cloud native serverless product with
AWS SAM
andAWS Lambda
. (Jump to section) - Wrapping it up with a conclusion.
An API or Application Programming Interface is a way of exposing your company’s digital services.
It is the layer through which your services communicate with other services.
API first development - Why, how and what
Great communication is key to great software engineering.
That also goes for applications and services. If your applications cannot communicate properly, you’ll never be able to expose the functionality that is key to a good product.
We also see the following trends in software engineering:
-
Shift towards the cloud.
Instead of big monolitic applications we are building lot’s of smaller (micro)services. All communication between those services goes through the API. -
Multiple frontend applications use the same backend.
Often these applications are created by separate teams. -
An API carries business value.
There is real money in your API.
Defining APIs gives us the opportunity to expose our application’s functionality and create bridges between our provider and consumers. The easier it is to integrate with your API, the higher the acceptance rate of consumers for your product will be.
In all of the cases above, there is value in good communication between services. And that’s why you should practice API first development. Put your communication first!
The first priority in your API first development story is a clear API definition
How do you practice API first development?
- Design your API before implementing it.
This will allow teams to develop their applications separately because they both know and understand how communication between the services will happen. The contract between services is set. - Understand that the API is the interface for your application.
It is the intersection where multiple services join hands to couple their functionality. - Visualize your API
An image says more then a thousand words. We’ll see how OpenAPI can help you with this.
API first development allows teams to develop separately against a common interface, the API.
Now that we understand the importance of and value of API first design let’s see how the Swagger/OpenAPI spec can help you with that.
Top-down vs bottom-up
API first development implies a top-down approach to build your API.
Basically there are two approaches:
- Top-down aka Design First
- Bottom-up aka Code First
To quote Swagger.io
:
Design First: The plan is converted to a human and machine readable contract, such as a Swagger document, from which the code is built.
Code First: Based on the business plan, API is directly coded, from which a human or machine readable document, such as a Swagger document can be generated.
In this blog post I am using the Top-down Design First approach to facilitate API first development.
In the last paragraph of this blog I’ll briefly show an example of a Code First approach with Springfox
.
API design: an example
Suppose that we, Ordina, are hosting a conference where multiple technical and agile sessions will be given. Users can check session information and register for sessions. The backend service is accessed by a web application and two mobile apps.
The applications are created by different teams and they all embrace the API first approach.
They read this blog and realised that by agreeing on the common interface first, they could develop separately without impacting each other. So hooray for API first development!
API first development with OpenAPI/Swagger
Let’s continue with creating the backend application.
The functional analysts and a couple of developers of the team are sitting together to agree on how the API should be defined.
- A client should be able to fetch all sessions via the API
- A client should be able to create a new session via the API
This is a crucial part of the API first mindset. We need to clearly define and communicate the API before starting to implement. Designing an API is easier when you can visualise the API. Let’s bring in the OpenAPI spec.
The OpenAPI specification allows you to define your API in a descriptive language (JSON or Yaml) and nicely visualise it
Let’s now use OAS to help us with our API First approach and design our API. Note that by OAS I mean OpenAPI Specification.
OAS stands for OpenAPI Specification (formerly known as Swagger Specification)
If you are confused about the difference between OpenAPI and Swagger, check out this page.
Time to introduce you to https://editor.swagger.io, a portal to visualise… Easy to use and offering all the functionality we need for this example.
I’ll keep it simple, we will create the OAS for exposing the endpoint to let consumers fetch the sessions of the conference.
The OAS allows you to use JSON or Yaml to describe your API.
Find the descriptive yaml via this gist.
As you can see from the example, the OpenAPI specification is very readable.
Even if it’s new to you, you should be able to deduct what is written in the yaml.
You like looking at raw yaml?
Sure you don’t!
There is a great visualisation to the right of it.
This clearly visualises what your API can do.
Clear visualisations mean clear communication.
The API in the example is small. When describing a whole real-world API, the file might become quite large. But that’s no problem. The OpenAPI spec allows you to split your definitions over multiple files which you can reference from within other files.
OpenAPI Takeaways: Easy descriptive language & great visualisations
You want to expose your beautiful visualisation to your clients.
They shouldn’t have to paste a yaml
file in a window of their browser all the time.
How do we do that?
Let’s find out next.
Hosting your visualisations
The API specifications should be easily accessible for you and your clients. The specification which you agreed upon, should be hosted somewhere for everyone to see. Sometimes companies have there own in-house tools to visualise OAS. If your company has no such tool there are plenty of other tools to visualize your API defined with OAS.
Choose a visualisation solution that allows you to show a diff between versions of your API
A couple of hosted solutions:
-
swaggerhub.com: Platform for API design and hosting by
SMARTBEAR
itself
-
Redocly: Redoc allows you to host via github pages. You can also host locally and integrate with Github pages for publishing your API. Use this Generator to create a repository for your API spec.
Integrating with Postman
You can create a working Postman collection from the OpenAPI spec
You don’t have to tell me how difficult it is to keep a Postman collection up to date with an evolving API! More so, you have to make sure that every member of your team has the latest version of your API collection.
Good news!
Postman can import a collection directly from the OAS.
In the Postman UI go to import
and import from raw text
.
Just like I did in the image below.
As you can see on the background of the image above, the request was correctly imported from the Swagger file.
OpenAPI Generator: generate API compliant code
Generate code that is compliant with your API spec with OpenAPI Generator
When you’ve agreed upon the specification of your API it is time to start implementing it! The specification is shared across the different teams and they can each start implementing separately.
Time to code!
If I write code, I might make mistakes.
So let’s generate code that is completely compliant with the specs.
OpenAPI Generator is a hugely popular repository on github. It allows you to generate code that is completely in line with your API specification.
On mac
you can just install the openapi-generator-cli
by installing it via brew.
brew install openapi-generator-cli
You can also checkout the github project, build it and use that jar.
You have the cli installed and created a directory which contains your api.yml file.
Let’s generate the code!
You could generate a whole project.
openapi-generator generate -i api.yml -g java
That would generate a whole Java project with a bunch of files. Let’s start a little smaller.
If you followed along clear the directory with
find . ! -name api.yml -delete
For starters we only want to generate the model classes:
openapi-generator generate -i api.yml -g java -Dmodels
Models here refer to your DTO’s (Data Transfer Objects) or Resources. These are different from your domain models or entity models.
Let’s see what we did here:
generate
Generate is the command that we give to the openapi-generator cli to instruct it to generate the code.
-i api.yml
The input file that contains our API specifications.
-g java
The generator to use. Here we specify that we want Java as output language.
-Dmodels
We are telling the generator to only generate the models for our API.
If you want help or you forgot one of the options you can look here or execute:
openapi-generator help generate
If you execute this command you’ll see that there are a lot more options. We could for example do the following
openapi-generator generate \
-i api.yml \
-g java \
-Dmodels \
-DmodelTests=false \
--model-name-suffix Dto \
--model-package "com.ordina.conference_app.model" \
-p useBeanValidation=true
This generates the java
models without creating test classes and puts them in a package com.ordina.conference_app.model
.
It suffixes them with Dto
since that’s what they are.
These classes are used to transfer data in and out of the application (Dto
aka Data Transfer Object).
Bean Validation
Keep your generated class files in sync with the requirements of the API specs by setting the useBeanValidation option to true.
In the last example I also specified a property useBeanValidation=true
.
Requirements specified in the API documentation like a required field are now translated to the code.
The getter
on the speaker
field in the SessionDto
class is now annotated with @NotNull
.
You can now use a framework like JSR 380
, known as Bean Validation 2.0.
, to validate input and output.
This is a Java
specific example, but the same will happen when you change to other languages by using eg. -g python
.
Now that we got acquainted with code generation I am going to show you how to include it in your build process.
Integrating Swagger/OpenAPI with your build process: maven or gradle plugin
There are maven and gradle plugins that support the openapi-generator
project.
(maven and gradle)
I started a maven project and included our api.yml
on the classpath.
Now it is a matter of configuring the openapi-generator-build-plugin
in our maven pom.xml
.
I want to configure it to behave the same way as the example above.
<plugin>
<groupId>org.openapitools</groupId>
<artifactId>openapi-generator-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<inputSpec>openapi.yaml</inputSpec>
<output>${project.basedir}</output>
<generatorName>java</generatorName>
<addCompileSourceRoot>true</addCompileSourceRoot>
<skipOverwrite>false</skipOverwrite>
<modelNameSuffix>Dto</modelNameSuffix>
<modelPackage>be.ordina.conference.api.model</modelPackage>
<generateModels>true</generateModels>
<generateModelTests>false</generateModelTests>
<generateModelDocumentation>true</generateModelDocumentation>
<generateApis>false</generateApis>
<generateSupportingFiles>false</generateSupportingFiles>
<library>jersey2</library>
<configOptions>
<dateLibrary>java8-localdatetime</dateLibrary>
<java8>true</java8>
<useBeanValidation>true</useBeanValidation>
<sourceFolder>src/java</sourceFolder>
</configOptions>
</configuration>
</execution>
</executions>
</plugin>
Important to note:
- The generated models will appear in the current module under
/src/java/be/ordina/conference/api/model
That is caused by the combination of multiple options:<output>${project.basedir}</output>
<sourceFolder>src/java</sourceFolder>
<modelPackage>be.ordina.conference.api.model</modelPackage>
- Only the API models will be generated, with markdown documentation and no test classes.
That’s a combination of:
<generateModels>true</generateModels>
<generateModelTests>false</generateModelTests>
<generateModelDocumentation>true</generateModelDocumentation>
<generateApis>false</generateApis>
<generateSupportingFiles>false</generateSupportingFiles>
Check out this gist for the xml.
Testing your API
Testing you API would normally involve setting up a larger integration test.
In the case of AWS Lambda this means that you’d have to deploy your application since you cannot run it locally (Not that easily at least).
Luckily we have set the useBeanValidation
property to true.
This allows us to write unit tests that validate the incoming and outgoing requests of our function.
After you deserialize the incoming request you can validate it against your API specs:
Set<ConstraintViolation<SessionDto>> violations = Validation.buildDefaultValidatorFactory().getValidator().validate(sessionDto);
If some violations are detected you can return them wrapped in a 400 response. You could easily check this functionality by writing a unit test that:
- checks that no violations are found in the case of a valid request body
- checks that violations are found in case a payload is sent which is not compliant with the API specs.
The same goes for the responses.
In your code validate the response against your API specifications by using the responseDto
that was generated from the specs:
Set<ConstraintViolation<CreateSessionResponseDto>> violations = Validation.buildDefaultValidatorFactory().getValidator().validate(response);
If this finds any violations throw a ConstraintValidationException
.
Again a unit test can validate that:
- no violations are found when the response is validated
- no exception is thrown
Serverless on AWS: OpenAPI, API Gateway Lambda and SAM
It’s fairly easy to create an API Gateway from an openAPI specification. In the API Gateway console under Create select Import from Swagger or Open API 3 and upload your specification.
Of course we want to use the specification programmatically. Suppose we create our backend service with AWS Lambda (serverless). I’ll be using AWS native tools and use SAM to deploy the Lambda functions and my API. SAM allows you to use an OpenAPI specification to create your API Gateway.
In your SAM template define the API Gateway resource by referencing your OAS.
ConferenceApiGateway:
Type: AWS::Serverless::Api
Properties:
StageName: dev
DefinitionBody:
Fn::Transform:
Name: AWS::Include
Parameters:
Location: ./openapi.yaml
In the Lambda function resource specify that the lambda should be triggered from this API Gateway.
GetAllSessionsFunction:
Type: AWS::Serverless::Function
Properties:
...
Events:
CreateSessionApi:
Type: Api
Properties:
RestApiId: !Ref "ConferenceApiGateway"
Path: /sessions
Method: GET
Add the x-amazon-apigateway-integration
extension in your api.yml
to specify how the api has to integrate with the backend Lambda service.
paths:
"/sessions":
get:
...
x-amazon-apigateway-integration:
type: "aws_proxy"
httpMethod: "POST"
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${GetAllSessionsFunction.Arn}/invocations
For a full template example including the Lambda resources, check out this gist.
Code First with Springfox
I promised you an example of a code first approach. Here I set up a Spring boot application with Springfox dependencies.
<dependencies>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
<version>${springfox.version}</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
<version>${springfox.version}</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-bean-validators</artifactId>
<version>${springfox.version}</version>
</dependency>
</dependencies>
Implementing the API and using the right annotations leads to an endpoint of your application on which your API spec is visualised: /swagger-ui.html
There is also an endpoint to download the Swagger / OpenAPI specification: api-docs
Conclusion
Here are some takeaways about API first development:
- Design the API before implementing it
- Visualize your API so that dependent teams can easily consult it
- API first development implies a top-down approach
- Swagger/OpenAPI can help you with API first development
Below the pros and cons of practising API first development using Swagger/OpenAPI.
Pros
- Strong tooling support — AWS, Postman, visualizing the API, generate skeleton classes, …
- Strong consistency between API spec and Web layer of the code
- Example support
- Documenting API descriptions is separated from code. Annotations are added to the generated code but you won’t be responsible for constantly updating them to keep documentation in sync.
- Functional analyst can assist with creating the API specs because it’s a human readable format
Cons
- No support for complex/variable request/response scenarios
- Little extra documentation can be added in the API specs
- If you add a new
enum
in the specification, your clients have to regenerate their code in order to be able to accept the enum. - Development can only start after API is designed
Resources
- https://scratchpad.blog/serverless/howto/configure-aws-api-gateway-with-swagger/
- https://github.com/jthomerson/cloudformation-template-for-lambda-backed-api-gateway-with-dynamodb
- https://medium.com/capital-one-tech/how-to-make-swagger-codegen-work-for-your-team-32194f7d97e4
- https://github.com/OpenAPITools/openapi-generator
- https://howtodoinjava.com/swagger2/code-generation-for-rest-api/
- https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#globals-section
- https://www.47northlabs.com/knowledge-base/generate-spring-boot-rest-api-using-swagger-openapi/
- https://swagger.io/blog/api-design/design-first-or-code-first-api-development/