UK-based, Civo, is a cloud-native service provider powered by Kubernetes. The company continues to grow, which means it needs a scalable operating platform and an accompanying storage vendor. It chose StorageOS, software-defined, cloud native storage for Kubernetes. Today, Civo has two super-clusters with more on the way this year. They both run Kubernetes as the base layer and StorageOS within that, providing persistent volumes. Andy Jeffries, CTO and Founder at Civo, explains what StorageOS brings to the table.
Civo provides one of the world’s fastest-to-deploy managed Kubernetes services. Using K3s, Civo customers can launch a Kubernetes cluster in around 90 seconds compared to the 20-30 minutes and substantial effort required with most services.
Originally founded in Hertfordshire in the UK, Civo now has a globally distributed team with employees in UK, America, India and Malaysia. The company’s target markets include small to medium development teams that need fast Kubernetes access for experimentation and testing, demos, learning and training environments and continuous integration or delivery (CI/CD). The company also supports individual developers who work for – or represent – larger enterprises, including Oracle, Walmart, Verizon, Microsoft and NetApp.
The challenge
Civo’s original platform was based on OpenStack and initially used software-defined storage from Ceph. However, the development team quickly realised that this combination wasn’t delivering as anticipated. Andy Jeffries, CTO and Founder, explained: “There were just too many maintenance and development issues at the platform and storage level. True, Ceph held many promises in terms of being a scalable solution.
“But it felt like we were battling constantly to maintain a perfect working state. Some disks would fill up, while other disks would be 60% empty – yet the whole cluster would go into a warning state, with alarm bells ringing everywhere. It wasn’t an easy or stress-free way to run a managed service business.”
Persevering with OpenStack, Civo moved away from open-source storage and tried NetApp, an all-flash solution. It was a move in the right direction, but the solution still required a lot of management and late-night calls when the storage failed or customer instances dropped into read-only mode. It was clear that the time had come to find a new operating platform and accompanying storage vendor. One that could support a growing managed services provider with reliable, scalable and self-healing cloud-native data availability.
The solution
Civo considered several storage vendors, including OpenEBS, Portworx, StorageOS and Rook. StorageOS was ultimately selected because it delivered on three essential requirements – performance, data safety and support.
“When we talk about performance and data safety, we’re talking about the strong replication, high availability and failover that StorageOS brings to the table,” said Jeffries. “StorageOS also demonstrated that its team understood the unique way cloud and managed service providers consume storage in terms of the vast number of volumes that are created and deleted on a regular basis – and the accompanying elasticity that’s required to make that happen.”
Today, Civo has two super-clusters with more on the way this year. One in London and one in New York. They both run Kubernetes as the base layer and StorageOS within that, providing persistent volumes. When a Civo customer launches a K3S cluster, the platform spins up a series of virtual machine disks with storage provided by StorageOS.
The benefits
Without the capability to persistently store data, clusters can fail and then customer applications can be affected. In the past, Civo was able to work around this issue by having regular I/O compute instances. If a customer needed a database or uploaded file storage, Civo had to manually create an instance and use that as the backend storage. With StorageOS, it’s a native part of the deployment and – as such – happens automatically. StorageOS also enables Civo to carry out maintenance and rolling upgrades of its Kubernetes clusters, thanks to its replication function and transparency when moving volumes between nodes.
“Ours is the kind of offering that tests the limit of IT infrastructure and container-based applications,” said Jeffries. “We’re always hitting system limits and bottlenecks – like distributed key value stores. StorageOS is always keen to work with us as we push these boundaries. Its team monitors how close we are getting to the limits and, before we hit them, offers advice on how we can extend the capability so that it doesn’t become a bottleneck.”
In addition, Civo’s development team has been impressed with the strong product roadmap that StorageOS has in place and its willingness to engage openly with future product developments, noting that StorageOS is always open to discuss and understand what features it might need in the future.
Jeffries said: “For a specialist managed service provider trying to break our way into a larger market segment, this kind of openness and transparency from a vendor is critical in planning our evolution and delivering new products.”
Civo has also written its own storage driver, which utilises StorageOS. This allows customers to create their own volumes of persistent storage. Jeffries provided some additional detail: “Using StorageOS in this way has allowed us to reduce the price of our Kubernetes clusters – offering customers smaller static disks on the instances and still giving them the ability to add any size of volume they want to them. This means customers now pay for what they use rather than resources they don’t need. To that end, you can buy a 1GB StorageOS volume attached to one of our Kubernetes clusters for as little as US$0.10 per month.
“As a self-funded start-up, our biggest challenge has always been managing the entire platform – including the storage component – with a relatively small team. While it’s true that we’ve grown over the last 18 months, we’re still a lean operation, so we must rely heavily on product automation and vendor support to make sure we continue to deliver to our customers. StorageOS continues to support us and more than pulls its weight in this regard. As such, we now consider the company to be a key technical partner and a critical enabler for our future growth and evolution.”
Jeffries told Intelligent SME.tech more about how the service has helped his company.
Can you give me an example of how a small or medium business development team would use your Kubernetes service?
They often use our offerings to bring their staff up to speed with using Kubernetes. Our recently released survey analysis (https://www.civo.com/blog/Kubernetes-in-2021) showed that the steep learning curve is one of the largest roadblocks to Kubernetes adoption. So, our speed of launch along with our open marketplace allows for staff to experiment and learn Kubernetes, while keeping costs down.
Tell me more about how StorageOS understands the unique way cloud and managed services providers consume storage.
A lot of cloud/container customers running persistent storage leave the storage in place for a relatively long time. Managed service providers such as Civo utilise persistent storage provided by StorageOS in a very different way with lots of volumes being created and destroyed each day as customers launch and destroy both Kubernetes clusters and persistent volumes within them. This frequency of resource creation and deletion can be very challenging in cluster-based systems.
Why does your kind of offering test the limits of IT infrastructure and container-based applications?
There are lots of communication issues with large clustered systems running on a hyper-converged infrastructure, such as handling the creation of volumes located across multiple nodes with relatively concurrent writes for data resilience. Adding our frequency of creation and deletion is a real challenge for all parts of our Kubernetes supercluster-based stack and the storage subsystem.
How well does StorageOS work with your team?
The StorageOS team has had a very close relationship with the Civo technical team since the very outset. Our first conversations were in person (this was pre-COVID) and it sent a team member to our site to work with us on a pre-order proof of concept. Since then, we regularly talk about issues, roadmap and the general cloud-native and storage landscapes and we’ve always felt that they are 100% ready to support us and work with us as we grow further.