This article will try to explain what Platform as a Service model is and why it is the best way to utilize FIYU Rapid Application Development platform for multiple projects. It will also go over motivation and challenges of creating a FIYU PaaS provisioning platform and requirements for its creation and explain why utilizing StackStorm automation platform was a natural choice to build upon.
PAAS concept generally
Since PaaS (Platform-as-a-Service) is mentioned often in this article, let’s quicky pin down exact meaning of the term PaaS for more precise understanding.
In software development, platform-as-a-Service is a cloud computing model where a third-party provider delivers hardware and software tools hosted on its own infrastructure to customers over the internet. By using PaaS, customers are typically freed from purchasing their own hardware and software to create their own development platform, which them saves energy, time, and money. In the case of FIYU, customers not only get a complete development platform, but also by using FIYU modules and libraries, development speed is enhanced, and total amount of time is further reduced.
FIYU architecture highlight
If this is your first FIYU-related article, you would probably want to know more about FIYU itself and its features. Feel free to submit a request for whitepaper at the bottom of the page.
There are also useful blog posts that can be found on this link: https://fiyu.app/blog/
Now let’s look at a very simplified FIYU environment architecture:
If you have any experience in DevOps and system administration, tasks that need to be executed when bootstrapping an environment like this quickly become obvious.
Postgres and Mongo databases need to be created, owned by dedicated database users with correct permissions. GitLab groups, users and repositories need to be created. The new Microtica project needs to be set up with a correctly created cluster connection. Project pipelines need to be created and connected to correct previously created GitLab repositories.
Customer should have a starting point when working with FIYU, so seed frontend and backend projects need to be generated according to customers input using npm for frontend and Maven archetype for backend.
After seed projects are built on Microtica, the resulting docker images need to be pushed to GitLab registry. FIYU services need to be defined on Microtica using the just built seed docker images and images of core modules. AWS deployment configuration needs to be generated.
After preparation steps, services need to be deployed to correct AWS cluster in specific order. Some validation should take place to check if everything is up and running. Once deployed, perhaps a new user needs to be set up with correct permissions on the newly created FIYU environment? Perhaps it is necessary to automatically monitor the new environment, send alerts in case of failures and calculate availability? Perhaps a summary email needs to be sent to customer’s email address at the end, containing all relevant information, login URLs, usernames, tokens, database information, etc.?
Sounds challenging?
Now imagine receiving ten emails from ten customers, each requesting one or more FIYU environments with a unique project name, organization name, email addresses, login and service URLs, usernames, and passwords. Let’s not forget that all the environments will have to be undone and all the resources recycled once projects are over for whatever reason.
It can easily take days or even weeks to manage environments by hand. Manual configuration is also intensive, time-consuming, repetitive, error-prone, tiresome and a great way to quickly lose an employee. There must be a better way.
FIYU enables faster software delivery, but we need a way to deliver FIYU faster.
If only we could accelerate software delivery by automatic FIYU provisioning.
Introducing FIYU PaaS provisioning platform
It was decided to invest development time in creating an automation platform for FIYU PaaS environment delivery. A general list of requirements for such a system would consist of:
- Flexibility – ability to utilize multiple programming languages, UNIX tools or include other software without restrictions to achieve desired functionality.
- Free / open source – no license fees.
- Standard development best practices – using git, CI/CD pipelines, testing, portability, modularity, code reusability and other coding best practices during development.
- Simple and efficient permanent storage – provisioning requires a significant number of parameters that should be easily manageable.
- User Interface – decent tools for system use and administration.
Writing a custom provisioning service from scratch might seem as a clean and compelling option, but reinventing the wheel would require time and resources we currently do not have available. Fortunately, there are existing open automation platforms that can be utilized free of charge.
Such platforms usually have all the basic functionalities like a web user interface, network APIs or means for creating and triggering a workflow solved and well tested. New platform users are then welcomed to customize and extend the existing platform code with proprietary modules to achieve their own desired business goals.
Introducing StackStorm
One of the open automation platforms that quickly ticked all the boxes from the previous chapter was StackStorm. Feel free to investigate at their website.
StackStorm is an open-source advanced event-driven automation platform trusted by enterprises and currently developed by the Linux Foundation.
Among its many great features, the most notable are great documentation, optional ready-to-use integration packs contributed by community, nice web user interface, advanced CLI support and many more.
StackStorm is written in the Python programming language, so Python has first-class support when extending the system with custom functionality. All existing Python modules available on the Python Package Index (PyPI), documented and tested and covering most of the tasks you could ever think of, can be easily installed on a server, and utilized to generate more compact code more quickly. Python’s modules system, OOP, exceptions, concise syntax, speed of development and other benefits are also utilized automatically during development.
In case Python is not your first weapon of choice, the system has support for executing bash scripts where it should be possible to automatically trigger binaries or start scripts written in any other programming languages.
StackStorm also has a built-in Redis-like datastore service that can be used extensively to store and fetch key-value pairs as needed.
StackStorm actions are reusable standalone pieces of functionality that can be directly invoked separately or connected into more complex workflows and executed sequentially. Workflows have branching options depending on the status of the previously finished action. Specifically, if the previous action failed, it is possible to break the standard action chain, execute some other workflow branch and exit. Thus, in case of an action failure, a descriptive email is automatically sent to administrators, the workflow is marked as failed and terminated.
Workflows keep track of progress, so it is possible to fix the cause of a failure and simply restart the failed workflow.
Workflows support variables so data can be shared among different actions. The datastore can also be used directly for this purpose when accessed directly within the action.
With a bit of effort, a new GitLab repository was created, hosting a new FIYU StackStorm pack and a continuous deployment pipeline was created to sync the repository with the StackStorm development server on every commit.
All systems involved in the FIYU PaaS provisioning fortunately have API endpoints.
A few months later, the initial version of the FIYU PaaS provisioning platform was released.
Using the FIYU PaaS provisioning platform, provisioning of a single new FIYU PaaS environment was reduced from days of hard work to 15 minutes of optional workflow progress tracking.
Deprovisioning workflow was also created to automate the deprovisioning of FIYU PaaS environments and recycling of the resources. It also saves administrators hours of repetitive clicking.
Utilization of a provisioning platform gave FIYU the additional speed it needed.
Conclusion
This article explains the motivation behind creating the FIYU PaaS provisioning platform. A simple overview of StackStorm features is given and it is explained why StackStorm automation platform was chosen as a building base for the FIYU PaaS provisioning platform. The new platform enabled quick spawning of complex FIYU PaaS environments.
