15 Nov 2023 An Introduction to KNIME Business Hub
KNIME Analytics Platform is an open-source platform for data science that helps individuals and organisations to make sense of data. Based on the premise that data science is a collaborative discipline, the thriving KNIME Community Hub is a place where KNIME users work together on thousands of data science solutions globally, sharing knowledge, building workflows, and integrating new tools.
So far, the free KNIME Analytics Platform has been complemented by KNIME Server, the commercial software for productionising workflows. Now, with the latest release, KNIME Server capabilities have been integrated into KNIME Hub and are offered as KNIME Business Hub. This upgrade offers a single environment for all data workers to securely collaborate and share best practices, as well as enabling the creation of scalable analytics services within the organisation.
KNIME Hub is now available online as a hosted and supported KNIME Community Hub, or it can be deployed as KNIME Business Hub within an organisation’s private infrastructure.
In this series of blog posts, we will explore many of the features of Business Hub, focusing on topics such as continuous deployment for data science, multi-node Business Hub scalability, and modernised reporting, as well as some enhancements that we have built on top of it to meet specific customer requirements, which we hope will be of interest to other KNIME Business Hub users.
In this article we are going to highlight the benefits of using the new KNIME Business Hub to scale data impact, but we’re also going to guide you through its use as either a user or an administrator, with a focus on testing its main features and comparing them to previous KNIME Server releases.
Why Use KNIME Business Hub?
Whilst KNIME Community Hub enables users to contribute to a broad range of data science solutions worldwide, to share knowledge, to collaborate on workflows, and to integrate new tools into the open-source KNIME Analytics Platform, the new KNIME Business Hub provides companies and organisations with a single environment in which all data workers can work together securely, exchange best practices, and deploy and monitor data science solutions.
With KNIME Business Hub it is now even easier for analytics communities to grow quickly within an organisation, with workers sharing and learning from each other’s solutions in KNIME’s no-code/low-code environment.
The integrated capabilities of KNIME Server and KNIME Hub allow enterprises to leverage the same principles that drive data learning and collaboration in the open-source community.
What is KNIME Business Hub?
KNIME Business Hub is a customer-managed KNIME Hub instance that lets you access Hub resources and customise specific features, as well as granting your employees access and the ability to manage certain resources and organise them into teams.
Installation:
KNIME Business Hub is hosted on a Kubernetes-embedded cluster, an open-source system for automating deployment, scaling, and management of containerised applications such as KNIME.
Kubernetes clusters are composed of pods, the logical entities where containers are deployed. To enable self-healing capabilities, Kubernetes regularly monitors the status of pods and their containers. If a container fails, Kubernetes automatically redeploys it to achieve the desired state and restore operations. For installation and day-to-day operations, kubectl can be set up and used to monitor the pods and their status.
Here at ClearPeaks, we have provisioned a single-node cluster on an Ubuntu Virtual Machine on Azure to deploy KNIME Business Hub for evaluation purposes. We employed the Replicated Kubernetes installer, based on the kURL open-source project maintained by Replicated, for the installation.
Installing the KNIME Business Hub embedded cluster with Replicated will enable your enterprise customers to easily manage, update, configure, monitor, back up, restore, and troubleshoot their KNIME Business Hub instance with the KOTS (Replicated Kubernetes Off-The-Shelf) Admin Console.
Usage:
Once you’ve got your licence and done the installation, KNIME Business Hub will be available in your company, providing you with the ability to perform tasks such as:
- Create and manage different teams, members, and spaces.
- Collaborate with your colleagues either in a team-owned public space or a private space.
- Upload items from KNIME Analytics Platform to KNIME Hub.
- Keep track of changes with versioning.
- Test workflow executions and manage different execution contexts.
- Create and share data apps, schedules, and API services.
- Manage different deployments.
All of these are covered in this article!
How to use KNIME Business Hub
As Global Hub administrator, with KNIME Business Hub you can create and manage different teams within your organisation, enabling groups of users to work on shared projects with specific Hub resources.
From a user perspective, Business Hub allows you to connect from KNIME Analytics Platform, manage teams if you are the team administrator, upload items to KNIME Hub, keep track of different versions, manage the workflow executions, and create and deploy data apps, schedules, and services.
User Management
Users are managed in the backend through the Keycloak instance embedded within KNIME Business Hub, so the procedure to designate a user as a Global Hub admin is carried out via the Keycloak Admin Console. Likewise, Keycloak is tasked with the creation of new users upon registration.
Once the KNIME Business Hub is up and running, if you wish to delete, filter, or view the list of users with access to your KNIME Business Hub instance based on their team, visit the Hub URL, sign in with the admin username and password created during the installation, then click on your profile picture, select Administrator, and select Users from the menu on the left.
Team Management as Global Hub Admin
Teams are first created by the KNIME Hub global administrator, who is responsible for assigning a user to the role of team administrator, and for allocating resources to that team. The team administrator will be able to start adding team members and create execution contexts that can be used by the team.
To create and manage teams, first click on Teams to get an overview of the existing teams and to manage them. Once redirected to the new team’s page, you will be able to either add members to the team or to change their role to, for example, promote a user to team administrator. To do so, click on Manage Team and enter the username of the user you want to assign as team administrator. If you want to delete the global admin from the team members list, click on the bin icon corresponding to that user. Moreover, as a team administrator, you will be able to change the name of the team.
On the Teams page there’s an overview of the resources allocated to the teams, their availability, and their current usage. Click on the three dots in the corner of the team you want to allocate resources to, and three options will be displayed: Manage resources will allow you to specify both the maximum number of members and execution vCore tokens allowed in a team, Manage members enables you to add members to a team or to change the team members’ roles, and Delete a team will delete all team resources, data, and deployments.
Connecting to KNIME Business Hub
You can connect to your KNIME Business Hub instance either from a standard web browser or from the KNIME Analytics Platform. Each created user is assigned to one or more teams so that they can work on projects together.
When you navigate to your KNIME Business Hub’s address via a web browser and log in with your credentials, you’ll land on the KNIME Hub homepage. If you’re using KNIME Analytics Platform, you can easily access your Business Hub instance by adding its mount point to the KNIME Explorer. Your user role determines the workflows you can execute, ensuring you only access those relevant to your team.
Additionally, the KNIME Hub view feature allows you to effortlessly switch between multiple KNIME Hub mount points. Just as with KNIME Hub, KNIME Business Hub lets you search for and drag and drop items such as nodes, components, workflows, and extensions.
User Team Roles
There are two types of roles a user can be assigned when part of a team: an administrator or a member. The team creator is automatically assigned an administrator role and can promote any of the team members to admins.
A Team Administrator can:
- Add/remove users from the team.
- Promote team members to admins.
- Create execution contexts.
A Team Member can:
- View the team page, with member list and spaces.
- Create, modify, and delete public and private spaces.
- Upload/download items to/from public and private spaces.
- Delete items from public and private spaces.
Team-owned Spaces
A space is a repository for a project and each team can own an unlimited number of both public and private spaces. In a team’s public space, the items stored there will be accessible to everyone with access to the Business Hub instance and will be visible as search results in KNIME Hub searches (only team members have upload rights to the team’s public spaces).
In a team’s private space, only the team members have read access to the items that are stored there, allowing KNIME Hub users who are part of a team to collaborate privately on a project.
If you go to your user profile and click on one of the teams you are assigned to, you will be able to create a new private or public space for the selected team. You can then change the name of the space and add a description. As team administrator you can also change the visibility of the space from private to public and vice-versa, or delete it.
Moreover, you can manage access to a specific space by clicking on the pencil icon. In the Manage space access side panel that opens, you can change other team members’ rights with respect to the items in the space by granting them Edit rights or View rights.
A team admin can also change the role of the team members. In your team’s profile, if you click on the Manage team button, a side panel will open where you will be able to see a list of the team members and their assigned roles. Here you can manage the team members by selecting the roles you want to assign to each user, adding a new member, or deleting an existing member.
Uploading Items to KNIME Business Hub
From the KNIME Analytics Platform you can upload items to your KNIME Hub spaces by dragging and dropping or by right-clicking on the item. Just like KNIME Hub, KNIME Business Hub allows you to move items within the Hub and to delete and download items from KNIME Hub. You can now also manage the download of workflows or components from KNIME Business Hub in the browser.
Versioning
Another new feature is the tracking of changes with versioning when uploading items to a space. You can create versions of the space so that you can go back to a specific saved version at any point in the future. Once a version of the space has been created, or a KNIME Analytics Platform workflow has been updated, new changes to the space will show up as unversioned changes. In the Space history panel, you can both navigate and download a space version or delete an existing one.
Execution of Workflows
In order to execute workflows, your team needs an execution context to be assigned, which provides dedicated execution resources with configured execution settings for the execution and deployment of workflows that are owned and managed by the teams.
The team admin can edit an existing execution context and create a new one by going to the Settings tab in the team profile page. Here you will see all your team’s available execution resources, how many vCores are available and in use for each execution context, the version of the executor, its memory usage, and the CPU load.
You can define your execution context with the following parameters:
- The name of the execution context.
- The name of the Docker image of the executor that you can obtain either from your KNIME Hub global administrator or as a public Docker image made available by KNIME.
- A list of the blacklisted nodes that should be blocked by the executor (you need to provide the full name of the node factory).
- The number of executors.
- The number of vCores per executor.
- The amount of RAM per executor (GB).
It is possible to run workflows that have been uploaded to a team-owned space if the space is provided with an execution context.
Unlike previous KNIME releases, if you want to run a one-off execution of a workflow uploaded to a space, perhaps for testing purposes, you can go to the workflow page in KNIME Hub and click on Run. Now you will be able to configure your ad hoc execution by selecting the execution context you want to use, and the workflow actions you want to apply. In addition, when you select Ad hoc execution in the menu, you can see an overview of all execution jobs that have been performed on a specific workflow.
Creating and Managing Deployments
After a workflow has been uploaded to KNIME Hub, different types of deployments can be created:
- Data Apps: Data Apps provide a user interface to perform scalable and shareable data operations such as visualisation, data import, export, and preview.
- Schedules: A workflow can be scheduled to run at specific times and perform specific actions based on the result of each execution.
- API services: A workflow can be deployed as a REST endpoint and be called by external services.
Once you’ve selected one of the deployment options mentioned above, different parameters such as the deployment name, the space version, the execution context, and the workflow actions will need to be specified.
In the Deployments tab in the right menu, you can see a list of deployments that have been created for a specific workflow. Moreover, you can see all the deployments that were created under a specific team on the team’s page.
You can also perform different actions for each deployment: you can run/open, manage access, and discard a deployment. After execution, if you expand the data app section in the deployments of the current workflow, you will see the status of the deployed execution. You can also activate or deactivate a schedule from the list of deployments available.
If you choose to manage access, a side panel will appear where you can add the name of the user you want to share the deployment with. You can share the deployment with users that are not members of your team too, and they will be able to see the deployments shared with them in their profile in the Shared with me section.
Another new function from KNIME Business Hub is the business portal made available to every registered user, so that consumers can see all the data apps shared with them, execute them at any time, and interact with the workflow via a user interface – all without the need to build a workflow or even know what is happening under the hood.
Enterprise Features
When comparing KNIME Server and KNIME Business Hub, it’s important to check which enterprise features KNIME Business Hub offers, so now let’s look at some of these features.
Scalability
By using the Kubernetes off the shelf (KOTS) installation approach, you can achieve a multi-node installation. If you want to have a Kubernetes cluster in two nodes, you can provision a second instance (making sure you have sufficient resources) and run an install command from the KOTS Admin Console.
The cluster management page from the KOTS Admin Console shows all the clusters and enables you to add more nodes by providing an install script and the required token to join the cluster.
Authentication
Enterprise authentication comes with Keycloak, an identity provider in itself. From the Keycloak administration page you can manage users and even add a provider user federation such as your own LDAP or Kerberos to integrate it with your KNIME Business Hub.
Security
KNIME Business Hub must be installed into an Ubuntu, Red Hat or Amazon Linux 2 instance with certain port(s) exposed to inbound traffic. Ports such as SSH, kubectl, and the KOTS Admin Console, should only be exposed to the IP address(es) of the system administrator(s) of the Linux instance.
During the KNIME Business Hub installation process, when first navigating to the KOTS Admin Console URL provided in the embedded cluster installation output, you will be asked to configure the Transport Layer Security (TLS). KNIME Business Hub uses a self-signed SSL/TLS Certificate to secure communications between your local machine and the Admin Console during setup. If required, you may also be prompted to provide your own TLS certificate to secure traffic to the Admin Console.
Migration
In order to assist in moving your workflows from KNIME Server to KNIME Business Hub, KNIME has developed several workflows to help you with the migration.
Conclusion
On the one hand, from an infrastructure point of view, the new KNIME Business Hub architecture is a huge step forward compared with KNIME Server, as it is now based on Kubernetes, a cloud-native architecture which can run on any cloud platform such as Azure, AWS, GCP, or even in your own data centre.
Additionally, since KNIME Business Hub is based on micro-services, it not only enables you to scale the system both up and down to meet your needs as your organisation grows, but it also provides significant reliability, stability, and durability, thanks to Kubernetes’ self-healing capabilities.
What’s more, from a user perspective, you can now completely isolate teams from each other, both in terms of computing and workflows, with the Teams feature. KNIME Business Hub allows team administrators to be responsible for a specific part of the Hub. Moreover, the versioning feature improves workflow and model management, ensuring that users can access and utilise the latest and most accurate information.
Overall, KNIME Business Hub offers the capability to collaborate seamlessly within your team and between teams, avoiding the need for all teams to use the same resources and log in to a common platform.
This is the first in a series of blogs dedicated to KNIME Business Hub, so stay tuned for future posts where we will look at specific topics and customisation options in more detail.
In partnership with KNIME, we work with our customers to provide innovative enterprise solutions and services that fulfil their data science needs. Contact us now and find out how our expertise here at ClearPeaks can help you in your Advanced Analytics journey!
