10 Aug 2022 BI Project Management Part 1 – Introduction and Different Strategic Approaches

The successful implementation of a business intelligence (BI) environment or project in any company, with the required quality whilst also meeting time and budget requirements, is a highly complicated task and, of course, a significant milestone in the company story. An understanding of the different methodologies available in BI project management and development is the key to success.

In this article we are going to explore different approaches, focusing on the Agile Scrum framework, comparing it with other more traditional options in software development projects.

Business Intelligence Roadmap

The first part of any BI implementation is the definition of the project roadmap, which will include all the steps necessary to implement BI in your company. The roadmap should consider the topics we can see below:

Different Implementation Strategies and Methodologies

Once the BI roadmap and the principal project details have been defined, the organisation and the implementation team need to decide on the project methodology they are going to use.

There are many different methods and approaches, and all of them have their own advantages and disadvantages; and your choice depends very much on the use case, the size of the project, the type of organisation, and the degree of uncertainty that there might be about the project, amongst other questions.

Below we can see an overview of the main methodologies and the advantages and disadvantages of each:

Waterfall Approach

The Waterfall methodology is a traditional method, based on a sequential development process in which progress flows through different phases (requirements definition, analysis, design, build, etc.). This methodology is based on documenting every step and executing every phase in advance, and each phase will have formal exit criteria which all the players involved must sign off before starting the next phase. The main consequence of this method is that the users can’t interact with the solution until the final part of the project (delivery and acceptance).

A detailed example of a standard project implementation using this methodology is the Kimball lifecycle definition (Ralph Kimball is one of the most relevant persons in the field of data warehousing and BI design and conceptualisation).

In this example we can clearly see that the execution of the project is linear and there are dependencies between different steps.

Incremental Approach

The incremental approach is similar to the Waterfall methodology, with the difference that the project is separated into a group of defined releases or deliveries (as defined by functional domain, priority, or any business grouping criteria).

Iterative and Agile Approach (Scrum)

Agile emphasises a strong relationship between end-users and the development team, increasing user adoption of the delivery and helping to limit scope to the most important functionality. In particular, the Scrum methodology allows us to obtain BI solutions faster, in smaller pieces, unlike an “all at once” deployment solution. The method is based on defining a slot of time (iteration) with a clearly defined and unalterable workplan at the beginning of every cycle. After every iteration the team contributes with an increase in the value of the product. A demo session with the end-users is required to obtain feedback, and then it’s passed to production if the results meet the end-users’ expectations.

Focusing on Agile Business Intelligence

The Agile methodology was developed in 2001, defining four core values in the Agile Manifesto:

• Value individuals and interactions over processes and tools
• Value working software over comprehensive documentation
• Value customer collaboration over contract negotiation
• Value responding to change over following a plan

Agile Business Intelligence is the application of the Agile approach specifically for Business Intelligence projects. The main benefits are:

• Put the focus on the people creating the environment, motivating and supporting the team to improve the results
• Encourage responsibility within the team and increase autonomy and transparency
• Better project dimensioning, reducing associated risks
• Improve prioritisation and decision-making processes
• Better user experience
• Reducing time to market
• Ensure that the product delivery meets the needs of the final users, which can change dynamically, or respond quickly when scope and circumstances change

The Scrum framework is based on an iterative process, where the product evolves across multiple and continued iterations, applying new functionalities or modifications in the architecture, data models, ETL process, visualisations, reports, dashboards, and other BI features that are added over time, depending on the users’ demands.

The method consists of following a very simple process, whose steps are to be repeated across the iterations of the project:

• Initial Planning or Product Backlog: At this stage, before starting the following N iterative phases, all the team involved defines a big picture map of the future project, without going into details. This phase helps us to:
• • Understand the business questions that we want to answer by implementing the BI system
  • Identify the data sources involved
  • Understand the expected artifacts to deliver the required end-user information (layout and frequency)
  • Train the team to understand the framework methodology

The purpose of this initial project backlog, defined with the customer, is to understand the most important requirements in order to minimise reworking and to define the basic approach for the product.

• Requirements & Planning: The initial phase of every iteration, consisting of defining and planning all the work that we will try to achieve during the iteration, prioritising items to adapt to time and budget constraints. One of the most common techniques here is to ask the users what functionalities they want to implement.

• Analysis & Design: When we have defined the iteration plan, we go to every user’s history to analyse and design the technical solution.

• Development: Here the development teams deliver a prototype to comply with the requirements of the stakeholders; this will be a cyclical incremental method.

• Testing & User Acceptance: After building the iterations, the release is prepared for testing and acceptance by the users. During this phase we will:
  • Finalise testing scenarios
  • Finalise all the documentation
  • Demonstrate the prototype to a small group
  • End-user training
  • Preparation for deployment into production

• Deployment: This is the final stage, where the users start operating the system with the valuable increments added with the last iteration. During this stage:
  • End-users will operate the system, dashboard, and reports
  • Identify defects in the system and potential areas for improvement; these changes will be added to the product backlog to be tackled in subsequent iterations

Agile Business Intelligence can utilise any number of complimentary, agile frameworks like:

• Scrum (to manage incremental product development processes)
• Kanban (for its continuous workflow management process)
• Test-Driven Development (as a mechanism to ensure quality control in the work cycle)
• Extreme Programming (to provide sustainable delivery within teams)

In our next post we’ll look in detail at how to use the Scrum framework in your projects!

Another important characteristic of the Agile Scrum methodology:

• Teams are cross-functional, where people with different and complementary skills work together. The team should contain all the key skills required to be able to deliver the project requirements. The benefits are faster delivery times, rapid response to new issues, and improved information-sharing across the organisation (as the tasks are shared across all team members).
• Teams and customers work closely together towards the desired products.
• Quality assurance and testing are a critical component of Agile Business Intelligence. Dedicated testers are embedded in the team to execute quality control tasks with a separation between the developers and the testers.

In the second part of this article we will focus further on the Agile approach, particularly Scrum. Stayed tuned!

In the meantime, if you’ve got any queries yourself about setting up an BI project, don’t hesitate to contact the experts!