FinBERT-demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have fine-tuned FinBERT pretrained model on several financial NLP tasks, all outperforming traditional machine learning models, deep learning models, and fine-tuned BERT models.\n",
    "\n",
    "All the fine-tuned FinBERT models are publicly hosted at Huggingface 🤗. Specifically, we have the following:\n",
    "- **FinBERT-Sentiment**: for sentiment classification task\n",
    "- **FinBERT-ESG**: for ESG classification task\n",
    "- **FinBERT-FLS**: for forward-looking statement (FLS) classification task\n",
    "\n",
    "*Note: the following code is for demonstration purpose. Please use GPU for fast inference on large scale dataset.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:18.151916Z",
     "start_time": "2022-06-02T02:14:14.375587Z"
    }
   },
   "outputs": [],
   "source": [
    "from transformers import BertTokenizer, BertForSequenceClassification, pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:18.168084Z",
     "start_time": "2022-06-02T02:14:18.154558Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'4.18.0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# tested in transformers==4.18.0 \n",
    "import transformers\n",
    "transformers.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sentiment Analysis\n",
    "Analyzing financial text sentiment is valuable as it can engage the views and opinions of managers, information intermediaries and investors. FinBERT-Sentiment is a FinBERT model fine-tuned on 10,000 manually annotated sentences from analyst reports of S&P 500 firms.\n",
    "\n",
    "**Input**: A financial text.\n",
    "\n",
    "**Output**: Positive, Neutral or Negative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:26.392588Z",
     "start_time": "2022-06-02T02:14:19.137488Z"
    }
   },
   "outputs": [],
   "source": [
    "finbert = BertForSequenceClassification.from_pretrained('yiyanghkust/finbert-tone',num_labels=3)\n",
    "tokenizer = BertTokenizer.from_pretrained('yiyanghkust/finbert-tone')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:26.716634Z",
     "start_time": "2022-06-02T02:14:26.395425Z"
    }
   },
   "outputs": [],
   "source": [
    "nlp = pipeline(\"text-classification\", model=finbert, tokenizer=tokenizer)\n",
    "results = nlp(['growth is strong and we have plenty of liquidity.', \n",
    "               'there is a shortage of capital, and we need extra financing.',\n",
    "              'formulation patents might protect Vasotec to a limited extent.'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:26.724473Z",
     "start_time": "2022-06-02T02:14:26.719579Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'label': 'Positive', 'score': 1.0},\n",
       " {'label': 'Negative', 'score': 0.9952379465103149},\n",
       " {'label': 'Neutral', 'score': 0.9979718327522278}]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ESG-Classification\n",
    "ESG analysis can help investors determine a business' long-term sustainability and identify associated risks. FinBERT-ESG is a FinBERT model fine-tuned on 2,000 manually annotated sentences from firms' ESG reports and annual reports.\n",
    "\n",
    "**Input**: A financial text.\n",
    "\n",
    "**Output**: Environmental, Social, Governance or None."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:33.840910Z",
     "start_time": "2022-06-02T02:14:26.726802Z"
    }
   },
   "outputs": [],
   "source": [
    "finbert = BertForSequenceClassification.from_pretrained('yiyanghkust/finbert-esg',num_labels=4)\n",
    "tokenizer = BertTokenizer.from_pretrained('yiyanghkust/finbert-esg')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:34.063044Z",
     "start_time": "2022-06-02T02:14:33.843588Z"
    }
   },
   "outputs": [],
   "source": [
    "nlp = pipeline(\"text-classification\", model=finbert, tokenizer=tokenizer)\n",
    "results = nlp(['Managing and working to mitigate the impact our operations have on the environment is a core element of our business.',\n",
    "               'Rhonda has been volunteering for several years for a variety of charitable community programs.',\n",
    "               'Cabot\\'s annual statements are audited annually by an independent registered public accounting firm.',\n",
    "               'As of December 31, 2012, the 2011 Term Loan had a principal balance of $492.5 million.'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:34.069712Z",
     "start_time": "2022-06-02T02:14:34.065220Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'label': 'Environmental', 'score': 0.9805498719215393},\n",
       " {'label': 'Social', 'score': 0.9906041026115417},\n",
       " {'label': 'Governance', 'score': 0.6738429069519043},\n",
       " {'label': 'None', 'score': 0.9960240125656128}]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## FLS-Classification\n",
    "Forward-looking statements (FLS) inform investors of managers’ beliefs and opinions about firm's future events or results. Identifying forward-looking statements from corporate reports can assist investors in financial analysis. FinBERT-FLS is a FinBERT model fine-tuned on 3,500 manually annotated sentences from Management Discussion and Analysis section of annual reports of Russell 3000 firms.\n",
    "\n",
    "**Input**: A financial text.\n",
    "\n",
    "**Output**: Specific-FLS , Non-specific FLS, or Not-FLS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:41.074693Z",
     "start_time": "2022-06-02T02:14:34.072041Z"
    }
   },
   "outputs": [],
   "source": [
    "finbert = BertForSequenceClassification.from_pretrained('yiyanghkust/finbert-fls',num_labels=3)\n",
    "tokenizer = BertTokenizer.from_pretrained('yiyanghkust/finbert-fls')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:41.243049Z",
     "start_time": "2022-06-02T02:14:41.077405Z"
    }
   },
   "outputs": [],
   "source": [
    "nlp = pipeline(\"text-classification\", model=finbert, tokenizer=tokenizer)\n",
    "results = nlp(['we expect the age of our fleet to enhance availability and reliability due to reduced downtime for repairs.',\n",
    "               'on an equivalent unit of production basis, general and administrative expenses declined 24 percent from 1994 to $.67 per boe.',\n",
    "               'we will continue to assess the need for a valuation allowance against deferred tax assets considering all available evidence obtained in future reporting periods.'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-06-02T02:14:41.249606Z",
     "start_time": "2022-06-02T02:14:41.245135Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'label': 'Specific FLS', 'score': 0.77278733253479},\n",
       " {'label': 'Not FLS', 'score': 0.9905241131782532},\n",
       " {'label': 'Non-specific FLS', 'score': 0.975904107093811}]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "transformers4.19",
   "language": "python",
   "name": "transformers4.19"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}