Language Model Integration - Plan.Net

How can the core4os framework be extended with a language model?

Plan.Net Journey is the originator of the open source framework core4. This is used for most products for customers, such as hosting marketing dashboards, media mix optimization, web apps, customer journey visualization tools and much more.

The framework already takes a lot of components off the programmer's hands - it includes a role concept, distributed job chains and much more.

In order to optimize the accessibility of the data in the dashboards, an integration of a Language Model (Generative AI) in core4 is planned.

One conceivable application is the chat with dashboards, where customers can ask follow-up questions about data visualizations and the Language Model answers them based on the underlying data source, e.g. detailed information about a specific target group and their media usage.

Another possible application is to support our developers by answering questions about core4 based on the existing documentation.

In concrete terms, this means extending the core4 framework with a language model. This also includes the connection of different data sources to provide the AI with information.

A student team in the master course deep learning worked on solutions with the support of their teacher and staff from Plan.Net Journey:

Overall, some key challenges of the project were solved and a first proof-of-concept was presented. For example, a Large Language Model (LLM) such as GPT-4 was given access to "new" knowledge defined by the students by using an embedding model in combination with a local vector store to answer questions about core4os. It also ensures that the LLM "remembers" the previous course of the conversation and can therefore respond to queries. However, the quality and reliability of the answers generated were still in need of improvement. The students saw potential for further work here, particularly in the preparation and processing of domain-specific knowledge.