Market data giant Bloomberg is set to capitalize on the growing interest in artificial intelligence by developing a large language model specifically for finance, known as BloombergGPT, which boasts 50 billion parameters.
Bloomberg has published a research paper outlining the creation of BloombergGPT, which has been trained on a diverse array of financial data to facilitate various natural language processing (NLP) tasks within the financial sector. The firm states that the model will enhance existing financial NLP applications, including sentiment analysis, named entity recognition, news classification, and question answering, while providing Terminal clients the ability to leverage the vast amounts of data generated in the market.
To create BloombergGPT, the engineering team utilized an extensive archive of 40 years of financial data to generate a comprehensive dataset comprising 363 billion tokens sourced from English financial documents. This internal data was supplemented with a public dataset of 345 billion tokens gathered from sources like YouTube and Wikipedia, resulting in a training corpus exceeding 700 billion tokens. For comparison, OpenAI’s ChatGPT, released in 2020, was trained on only 500 million tokens.
From this extensive training corpus, Bloomberg’s team focused on a subset to train a decoder-only causal language model with 50 billion parameters. Gideon Mann, head of Bloomberg’s ML product and research team, emphasizes that “The quality of machine learning and NLP models comes down to the data you put into them.” He notes that the 40 years of curated financial documents allowed the team to build a large and clean, domain-specific dataset for training an LLM ideally suited for financial applications.
According to Mann, BloombergGPT significantly outperforms existing open models of similar size on financial tasks while maintaining competitive performance on general NLP benchmarks. Shawn Edwards, Bloomberg’s chief technology officer, highlighted the advantages of generative LLMs, such as few-shot learning and text generation, stating that the development of a financial-focused LLM presents tremendous value. He added that BloombergGPT will enable the firm to explore new applications while providing superior performance and faster implementation compared to custom models for each use case.