How to prepare and move your analytics and machine learning projects to the hybrid cloud – RTInsights

Vertica SQL database and in-database machine learning solutions support the entire predictive analytics process with massively parallel processing and a familiar SQL interface.

After a brief dip due to the impact of the pandemic, business analytics services returned to double-digit growth in 2021 and 2022, according to IDC. Why? There is a great need to improve business results using insights from analytics and other techniques, including machine learning.

What most companies find when taking on new analytics and machine learning (ML) projects is that their current infrastructure isn’t up to the task. Projects typically require compute and storage capabilities that aren’t typically available in most businesses. And in many cases, they need access to new analytics and ML tools and technologies.

The situation is a perfect storm for the hybrid cloud. The hybrid cloud gives businesses the flexibility to rapidly scale compute and storage infrastructure. It also enables the ability to mix and match, which means a business can selectively use the cloud or stay on-premises for different aspects of its workloads and data as needed. And they change the mix over time to meet changing needs.

Decide on capabilities, technologies, etc.

Many variables come into play when doing analytics and ML projects. Adequate compute, storage, and tools must be in place to achieve a project’s business goals. With the hybrid cloud, many options are available.

Take compute resources. If a business has made a substantial investment in high-performance on-premises computing capabilities, it can retain that investment and use the systems to analyze data on-premises or using cloud services.

On the other hand, if a company does not have the existing compute infrastructure, a hybrid cloud approach to analytics and ML allows it to take advantage of cloud compute instances and resources. This saves the huge CapEx expense that would be required to install features on-premises. And it gives the company access to newer technologies such as GPU and ARM processors that it may not have the experience to use. Such an approach also helps a company move projects forward much faster than would be the case when building on-premises compute infrastructure.

There are similar issues with storage where hybrid cloud can play an important role. Again, a company may have an existing on-premises database, their CRM or ERP system, for example. He may want to run sophisticated analytics on this database to improve operations or improve the customer experience. With a hybrid cloud approach, data can stay on-premises. And scans can be run on-premises or using a cloud service.

Alternatively, a company may have a large cloud database or a third-party cloud database that they want to use in an analytics project. With hybrid cloud, the database can stay where it is, hosted by a vendor, and analytics can be run on-premises or using a public cloud service.

Hybrid provides the flexibility needed today

The consistent theme with hybrid is that enterprises can keep what works, keeping workloads and storage where they are, while getting the insights their analytics projects are designed to deliver.

Why? Modern analytics and ML projects must deal with variables in compute, storage, and tools. That alone is hard enough, but there’s still more to consider. Perhaps the most challenging aspect of many projects today is that requirements change significantly over the life of a project.

For example, a company may need vast compute and storage resources to train a machine learning application. In a typical scenario, a large image dataset may be used to train the application. But once the model is trained, much less computing and storage capacity is needed. In traditional approaches, a company would have built the compute and storage infrastructure to complete the initial phase of the project. And then this ability would remain inactive after the training is completed. The hybrid cloud offers the flexibility to scale using public cloud services during this initial phase. And when the work is done, instances and cloud resources can be reduced.

Additional Factors to Consider

A particularly new area of ​​interest that lends itself well to a hybrid approach is the growing use of external data. Many companies now analyze their own datasets, but then supplement this analysis over time with third-party data. A hybrid cloud approach allows this data to be used when needed to improve the quality of analyses.

Examples abound in many areas. For example, a financial services organization may use third-party data to better understand its audience and target potential customers. Or a healthcare organization can combine anonymized patient records with data from a fitness app provider to better estimate the effectiveness of a program based on how well a person is exercising. exercise.

An additional aspect to consider is the analysis and ML software and tools themselves. If a company has invested heavily in such solutions, they can keep them and run them where they work best. But many companies are starting from scratch with modern analytics and ML projects. In such cases, they may not have the in-house expertise to select, deploy, manage and use these tools. In such cases, public cloud analytics and ML offerings can help.

For example, modern databases offer more advanced features, tools, and support than many on-premises technologies. Many large database vendors and cloud providers offer many analytics and ML solutions. And more importantly, they have vast ecosystems that can greatly help in accelerating projects. For example, many have add-on tools to ingest and prepare data for analysis. And some have extensive collections of tools that allow a company to build end-to-end hybrid cloud analytics pipelines.

Teaming up with a technology partner

A good example is the solution from Vertica, a business unit of Micro Focus. Vertica SQL database and in-database machine learning solutions support the entire predictive analytics process with massively parallel processing and a familiar SQL interface.

The availability of machine learning in the database provides several benefits. First, data scientists don’t have to move data around and wait for results when the analysis is run on less capable computing platforms. This alone saves time. But with machine learning capability in the database, processing power can be scaled as needed.

One last word

The one constant thing in modern analytics and ML projects is change. Everything is subject to change. Business goals often change over time. Compute and storage requirements change over time. New data is frequently added to the mix. New computing technologies and analysis methods become available over time. The hybrid cloud offers many benefits for analytics and ML, and it is well suited to meet varying demands over time.

By using a solution that converges databases and analytics through in-database machine learning, companies eliminate the need to move and prepare data. They can then create concise end-to-end workflows to operationalize predictive analytics. This is a real benefit of database convergence and machine learning.

Learn more: https://www.vertica.com/product/database-machine-learning/

Read the other blogs in this series: