Text As Data - Lecture 1

Use in economics and pre-processing

Machine Learning and Big Data

Vincent Bagilet

Introduction

Text as Data

  • A plethora of text data

  • Nowadays, we have both the technology and methods to study them massively

  • Text data is unstructured:

    • Basically not sorted in a sort of “table-like” format

    • Info we want mixed with info we do not want

    • Need to throw away some info ⇒\Rightarrow select what to keep

  • Text data is very high-dimensional (nvocabnwords in doc)\left( n_{\text{vocab}}^{n_{\text{words in doc}}} \right)
  • We often want to relate text data to metadata

    • eg who, when, on what topic?

Think About Your Own Question

  • What type of text data? What source?

  • How would you get the data?

  • Which research question?

  • How would you go about studying this?

Outline and Resources

Housekeeping




Assignment

  • Exercise to do for the end of next week

  • You can start today, even if we will not have covered everything

  • Available on Portail des études


Material

Objectives

  • Why is text data useful in economics?

  • What is a typical empirical workflow?

  • How to concretely implement these analyses in Python?

  • What are recent developments in the field?

  • Link this section to the rest of the class

Relation to the rest of the class

  • More on the data analysis and big data side than on the ML one

  • Here, focus on pre-processing steps, ie prepare the data to use it in:

    • A ML algorithm
    • An econometric analysis
  • Can use the tools and algorithms you saw in the rest of the class on text data

    • Examples?

Outline

  1. Introduction
  2. Applications in economics
  3. Workflow for analysis
  4. Pre-processing
  5. Representation
  6. Dictionary based methods
  7. Machine learning methods
  8. Introduction to deep learning methods and LLMs

Economics Resources

Elliott Ash’s Text as Data class

Python Resources

R Resources

Applications in Economics

Measuring Document Similarity

  • What?

    • Make pairwise document comparisons

    • Forming clusters of related documents

  • Examples

    • Cagé, Hervé, and Viaud (2020) use the distance between online news articles and social media posts to group items into common stories
    • Biasi and Ma (2022) build a measure of syllabi’s distance from frontier knowledge (academic articles) and then relate this metric to socio-economic variables
  • How?

    • Represent documents in some sort of vector form and then compute cosine similarity

    • There are different ways of defining vectors

Concept Detection

  • What?

    • Detect the presence of a concept
  • Example

    • Djourelova, Durante, and Martin (2024) interpretable news topics using CorEx
  • How?

    • Dictionary methods / pattern matching

    • Topic models to identify latent concepts

    • Distance between documents and dictionaries

    • Supervised (human classification) learning problem

    • BERT/GPT type of models

Relation Between Concepts

  • What?

    • How are concepts related?
  • How?

    • Co-occurrence of dictionaries

    • Word Embedding Association Test (WEAT):

      • Picks two terms at each end of the spectrum (eg rich and poor) and compute the cosine similarity of each term of interest with these “extreme” terms
  • Examples

    • Ash, Chen, and Ornaghi (2024) on gender attitudes of individual US judges

    • Kozlowski, Taddy, and Evans (2019) locate words on meaningful dimension, eg, rich/poor (this has a feel of Bourdieu’s social space)

Associating Text with Metadata

  • Impute an outcome of interest to other documents

  • Supervised learning

  • eg have some data on party label but not for everyone

  • Example

    • Gentzkow and Shapiro (2010) build a model for party label using US Congressional speeches. Then use it to predict political bias of newspapers

Workflow for Analysis

How are text data structured?

  • Text data is a sequence of characters called documents

  • A corpus is a collection of documents

  • The unit of analysis (the “document”) depends on the question:

    • Fine enough to fit relevant metadata variation

    • Not unnecessarily fine to reduce dimensionality

From text to a usable format

  • Text data is very highly dimensional

  • Need to transforming data into a useful representation for modeling

  • Encode text data into numeric arrays:

    • Tokenization and word counts

    • Document-Term Matrix

    • Latent Semantic Allocation

    • Word embeddings

  • Both for computational reasons and to have objects we can manipulate and do computations on

Steps of Text Analysis

  • Gentzkow, Kelly, and Taddy (2019) summarize a text analysis in three steps:

    1. Represent raw text DD as a numerical array CC

    2. Map CC to predicted values V̂\hat{V} of unknown outcomes VV

    3. Use V̂\hat{V} in subsequent descriptive or causal analysis

Workflow, rephrased

  1. Collecting data
  2. Pre-processing (stemming, lemming, etc)
  3. Data transformation (from pre-processed text to numeric arrays)
  4. Analysis and modelisation
  5. Validation
  6. Use in econometric analyses

Type of Analyses


  • Dictionary Methods: rely on pre-defined lexicons and info associated with specific keywords

  • Rule-Based Methods: use predefined rules and patterns (eg RegEx)

  • Machine Learning Methods: methods leveraging lightweight ML models

    • Supervised ML
    • Unsupervised ML
  • Deep Learning Methods: methods leveraging neural networks

  • Off-the-shelf Large Language Models (LLM) based methods


There is an important interpretability-flexibility trade-off

Gathering data

Common Text Data Sources

  • Political economy: news articles, parliamentary debates, speeches, social network posts, party manifestos, press releases, etc

  • Economic history: correspondence, institutional documents, books, etc

  • Labor and Industrial Organisation (IO): job adds, product descriptions, etc

  • Finance and macro: earnings conference calls, central bank speeches, etc

Where to find data

Overview of web scraping

  • The overall idea is to write algorithms that:

    1. Browse a website and download relevant html pages

      • Main challenge: identify the relevant pages
    2. Transform the html pages into a data frame containing text data

      • Typical structure: date column, title column, author column, text column, etc
      • Main challenge: identify the right tags
  • Not limited to text data: you can retrieve anything that is online

HTML pages




  • HTML: HyperText Markup Language

  • Structure of an example html page

<html>
<head>
  <title>Page title</title>
</head>
<body>
  <h1 id='first'>A heading</h1>
  <p>Some text &amp; <b>some bold text.</b></p>
  <img src='myimg.png' width='100' height='100'>
</body>

Webscraping concretely

  • There are libraries for web scraping with useful documentation:

  • Basic pages are static and easy to scrape

  • More complex to scrape dynamic pages

    • Use Selenium
    • It opens a browser and can be used to automate tasks

A few webscraping tips


Tips

  • Before scraping, look at whether someone did not already scraped what you want to scrape (eg for IMDb)

  • To identify relevant pages use the site sitemap (website.com/sitemap.xml)

  • Use ChatGPT or Claude for scraping

  • Use SelectorGadget or tools from the developer panel of your browser

  • Save html pages before processing them

  • Read the robots.txt page to learn what you can and cannot do

Optical Character Recognition (OCR)

  • Some data is either not digitized or digitized but in text/PDF format

  • Use software to transform it to text data

  • Tesseract and its Python and R wrappers

  • When PDFs are digitally made, you can use a text extraction software to directly retrieve the text from the metadata

Pre-processing

Goal

  • Going from raw text to something usable in analysis

  • ↘\searrow the dimensionality by ↘\searrow the size of the vocabulary

  • Identify some words as the same

  • Some information is useful, other less:

    • What to keep?

    • What format?

Tokens

  • Tokens: a meaningful unit of text, such as a word, a character

  • In NLP, often split data into tokens

  • Example from the French parliament: “Ne fumez pas la moquette”

  • Words: {Ne, fumez, pas, la, moquette}

  • n-grams (here 2-grams): {Ne fumez, fumez pas, pas la, la moquette}

  • Sentences

import nltk

# Run only once
nltk.download('punkt') 
nltk.download('punkt_tab')

sentence = "Ne fumez pas la moquette"
tokens = nltk.word_tokenize(sentence)

tokens


['Ne', 'fumez', 'pas', 'la', 'moquette']

Pre-processing Choices

  • Pre-processing choices often affects the results:

What To Keep?

  • Not all data is useful

  • Sometimes, may remove useful information: eg “happy” vs “not happy”

  • Remove capitalization?

  • Punctuation?

  • Numbers?

  • Stopwords?

    • Words that carry little information

    • How do you define this set?

Capitalization and punctuation


  • Often uninformative

  • But sometimes important:

    • Sentence splitting

    • Part-of-speech tagging

    • Named entity recognition

    • Text generation

  • Option: keep caps not at the beginning of a sentence


tokens_lower = [word.lower() for word in tokens]
print(tokens_lower)
['ne', 'fumez', 'pas', 'la', 'moquette']

Stop words

  • We have seen that might create noise but may also carry meaning

  • How to define stopwords?

  • Can use existing lists

stopwords = set(nltk.corpus.stopwords.words('french'))

print(stopwords)
{'ta', 'eusse', 'lui', 'une', 'ou', 'sont', 'soyez', 'avez', 'nos', 'vos', 'mes', 'n', 'as', 'étés', 'te', 'es', 'eurent', 'fusse', 'la', 'serons', 'était', 'aurons', 'aurait', 'étées', 'êtes', 'ne', 'eussiez', 'ses', 'se', 'eux', 'aurai', 'ayons', 'eut', 'eusses', 'que', 'eue', 'ayants', 'aviez', 'eu', 'le', 'serais', 'même', 'avions', 'eût', 'fûmes', 'avec', 'seras', 'soit', 'aies', 'étaient', 'eussent', 'les', 'auriez', 'étais', 'fussent', 'ayez', 'avons', 'par', 'aura', 'avaient', 'm', 'vous', 'qu', 'ma', 'fusses', 'avais', 'fus', 'votre', 'du', 'pour', 'ayant', 'dans', 'l', 'suis', 'ayante', 'ont', 'aient', 'nous', 'sa', 'ton', 'je', 't', 'd', 'étiez', 'étions', 'me', 'pas', 'aurions', 'notre', 'étée', 'serions', 'aurais', 'soyons', 'auraient', 'qui', 'serai', 'eus', 'furent', 'c', 'ai', 'tes', 'eues', 'été', 'ils', 'en', 'fûtes', 'son', 'fut', 'ait', 'de', 'moi', 'y', 'serez', 'seront', 'il', 'seriez', 'et', 'serait', 'j', 's', 'fussiez', 'soient', 'des', 'elle', 'eûtes', 'on', 'aie', 'est', 'auras', 'tu', 'étantes', 'étant', 'aux', 'mon', 'au', 'étante', 'fussions', 'sois', 'sera', 'aurez', 'avait', 'sommes', 'auront', 'toi', 'sur', 'eûmes', 'étants', 'seraient', 'leur', 'un', 'ces', 'ayantes', 'mais', 'ce', 'à', 'fût', 'eussions'}
tokens_filtered = [word for word in tokens_lower if word not in stopwords]
print(tokens_filtered)
['fumez', 'moquette']
  • Which list? How to define it?
  • Build one

    • By hand, starting from existing ones

    • Remove words that appear the most frequently, eg by Inverse Document Frequency (idf):

idf(term)=ln(ndocumentsndocument containing term) idf(term) = \ln \left(\dfrac{n_{\text{documents}}}{n_{\text{document containing term}}}\right)

  • idf decreases the weight of commonly used words and increases that of words that are rarely used in the corpus

  • tf-idf: multiplying the term frequency (tf) and its idf

Stemming/Lemmatization

  • Find the stem of the word:

    • Ruled based

    • Produces grams that are not actual words

stemmer = nltk.stem.snowball.SnowballStemmer(language="french")
tokens_stemmed = [stemmer.stem(word) for word in tokens]
print(tokens_stemmed)
['ne', 'fum', 'pas', 'la', 'moquet']
  • Lemmatization:

    • Similar but with semantic rules

    • Produces actual words but sentences that do not mean anything

import spacy
nlp = spacy.load("fr_core_news_sm")
doc = nlp(sentence)
lemmas = [token.lemma_ for token in doc]

print(lemmas)
['ne', 'fumer', 'pas', 'le', 'moquette']

Transforming your data

From text to numbers

  • Transform data to be able to use it in algorithms

  • Main data structures for documents in NLP:

    • As raw text
    • As a sequence of tokens for each document
    • As a vector, stored in a matrix (document-term matrix, embedding matrix)

Occurrences and counts


  • Raw count of occurrences of each term by document

  • Term Frequency (tf) of word ww in document dd (weight by the length of the document):

tfwd=Number of occurences of w in dTotal number of tokens in dtf_{wd} = \dfrac{\text{Number of occurences of w in d}}{\text{Total number of tokens in d}}

  • Account for the specificity of the term for the document: tf-idf (multiplying the term frequency (tf) and its idf)

idf(term)=ln(ndocumentsndocuments containing term) idf(term) = \ln \left(\dfrac{n_{\text{documents}}}{n_{\text{documents containing term}}}\right)

  • idf decreases the weight of commonly used words and increases that of words that are rarely used in the corpus

  • Can also use other transformations: dummy indicator of presence of a word in a document, log counts, etc

Matrices


  • Then create Document-Term Matrices that represent each document and each word in the vocabulary (ie in the corpus)
economy policy growth
Doc1 3 1 0
Doc2 0 2 4
Doc3 1 0 1
Doc4 2 3 1
  • These are bag-of-words (BoW) approaches:

    • Put all the term in a document together, regarding of their order (and count them)
    • Loose information about order between terms

Tf-idf implementation

import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer

assemblee_2018 = pd.read_csv("../../data/assemblee_2018.csv")

assemblee_small = assemblee_2018.head(20)
assemblee_small = assemblee_small[['speaker_name', 'date', 'text']]

vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(assemblee_small['text'])
X
<20x254 sparse matrix of type '<class 'numpy.float64'>'
    with 400 stored elements in Compressed Sparse Row format>
pd.DataFrame(data = X.toarray()).style.set_table_attributes('style="display:block; max-height:300px; overflow-y:auto;"')
  0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253
0 0.000000 0.000000 0.204050 0.000000 0.204050 0.204050 0.204050 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.204050 0.000000 0.000000 0.204050 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.204050 0.204050 0.000000 0.000000 0.224947 0.000000 0.000000 0.119645 0.000000 0.000000 0.000000 0.000000 0.000000 0.239290 0.000000 0.000000 0.204050 0.204050 0.000000 0.000000 0.000000 0.000000 0.148261 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.112474 0.000000 0.000000 0.000000 0.204050 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.161847 0.000000 0.000000 0.204050 0.000000 0.000000 0.358935 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.204050 0.000000 0.148261 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.204050 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.204050 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.204050 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.366948 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.366948 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.244774 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.366948 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.417454 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.230103 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.417454 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.366948 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
2 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.361225 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.240957 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.410943 0.000000 0.325950 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.240957 0.000000 0.226514 0.226514 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.361225 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.276232 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.410943 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
3 0.000000 0.234636 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.234636 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.387998 0.000000 0.000000 0.137579 0.000000 0.000000 0.000000 0.000000 0.000000 0.137579 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.170485 0.000000 0.000000 0.000000 0.000000 0.157720 0.000000 0.258665 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.234636 0.000000 0.000000 0.000000 0.234636 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.258665 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.186107 0.000000 0.000000 0.000000 0.234636 0.000000 0.170485 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.186107 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.234636 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.234636 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.234636 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.206248 0.000000 0.000000 0.000000 0.000000 0.206248
4 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.451358 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.451358 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.769774 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
5 0.220997 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.220997 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.175289 0.121815 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.220997 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.148552 0.000000 0.121815 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.220997 0.000000 0.220997 0.175289 0.220997 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.121815 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.148552 0.000000 0.220997 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.175289 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.220997 0.000000 0.220997 0.000000 0.000000 0.000000 0.220997 0.000000 0.220997 0.000000 0.220997 0.000000 0.000000 0.220997 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.220997 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.194259 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.220997 0.220997 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
6 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.505812 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.862644 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
7 0.000000 0.000000 0.000000 0.102046 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.102046 0.102046 0.000000 0.080940 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.102046 0.000000 0.000000 0.000000 0.000000 0.089700 0.000000 0.074146 0.089700 0.000000 0.000000 0.000000 0.204092 0.102046 0.000000 0.089700 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.102046 0.000000 0.000000 0.102046 0.000000 0.000000 0.102046 0.000000 0.000000 0.000000 0.161881 0.281242 0.000000 0.089700 0.179504 0.102046 0.000000 0.000000 0.102046 0.000000 0.239339 0.000000 0.000000 0.000000 0.000000 0.102046 0.000000 0.000000 0.102046 0.148292 0.000000 0.000000 0.000000 0.000000 0.068594 0.119670 0.000000 0.089700 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.080940 0.000000 0.161881 0.000000 0.000000 0.000000 0.074146 0.000000 0.000000 0.000000 0.102046 0.000000 0.000000 0.102046 0.000000 0.000000 0.000000 0.119670 0.000000 0.168745 0.168745 0.000000 0.080940 0.000000 0.102046 0.000000 0.102046 0.000000 0.000000 0.000000 0.000000 0.102046 0.080940 0.000000 0.000000 0.000000 0.000000 0.000000 0.074146 0.000000 0.000000 0.000000 0.089700 0.000000 0.000000 0.000000 0.000000 0.089700 0.102046 0.000000 0.000000 0.000000 0.074146 0.000000 0.000000 0.000000 0.000000 0.102046 0.000000 0.000000 0.000000 0.102046 0.102046 0.000000 0.000000 0.068594 0.000000 0.000000 0.000000 0.161881 0.102046 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.089700 0.089700 0.000000 0.068594 0.204092 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.102046 0.000000 0.102046 0.089700 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.102046 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.068594 0.000000 0.089700 0.000000 0.102046 0.102046 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.102046 0.204092 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.168745 0.102046 0.000000 0.000000 0.000000 0.000000 0.000000 0.161881 0.089700
8 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.388596 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.662737 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.365305 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.525667 0.000000
9 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.128843 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.113255 0.128843 0.000000 0.000000 0.102195 0.000000 0.000000 0.128843 0.000000 0.128843 0.000000 0.113255 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.128843 0.000000 0.000000 0.000000 0.000000 0.128843 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.284077 0.000000 0.000000 0.075547 0.000000 0.000000 0.000000 0.000000 0.000000 0.151095 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.093617 0.000000 0.000000 0.128843 0.000000 0.086607 0.075547 0.071019 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.204391 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.102195 0.000000 0.000000 0.000000 0.093617 0.000000 0.000000 0.128843 0.000000 0.204391 0.000000 0.000000 0.000000 0.128843 0.000000 0.302189 0.000000 0.071019 0.071019 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.113255 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.128843 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.113255 0.093617 0.128843 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.128843 0.000000 0.128843 0.000000 0.000000 0.102195 0.000000 0.000000 0.128843 0.000000 0.128843 0.000000 0.000000 0.000000 0.113255 0.000000 0.000000 0.086607 0.000000 0.113255 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.128843 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.128843 0.086607 0.128843 0.113255 0.000000 0.000000 0.000000 0.000000 0.128843 0.000000 0.000000 0.000000 0.128843 0.000000 0.257687 0.000000 0.128843 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.128843 0.000000 0.000000 0.000000 0.000000 0.000000 0.284077 0.000000 0.000000 0.000000 0.000000 0.000000 0.113255 0.000000 0.000000
10 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.707107 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.707107 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
11 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.110404 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.110404 0.000000 0.139192 0.000000 0.000000 0.000000 0.101136 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.230171 0.000000 0.122352 0.163231 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.093564 0.000000 0.153447 0.122352 0.000000 0.000000 0.000000 0.000000 0.122352 0.000000 0.000000 0.000000 0.000000 0.110404 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.122352 0.000000 0.122352 0.139192 0.000000 0.101136 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.076724 0.139192 0.110404 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.122352 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.000000 0.000000 0.110404 0.000000 0.278385 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.000000 0.000000 0.101136 0.000000 0.000000 0.000000 0.139192 0.000000 0.122352 0.000000 0.139192 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.139192 0.187128 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.139192 0.000000 0.139192 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.093564 0.000000 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.122352 0.000000 0.000000 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.000000 0.230171 0.000000 0.000000 0.000000 0.139192 0.000000 0.000000 0.110404 0.000000
12 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.197184 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.149588 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.159125 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.159125 0.149588 0.000000 0.000000 0.271382 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.215254 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.318250 0.000000 0.149588 0.149588 0.000000 0.215254 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.430508 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.238549 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.271382 0.000000 0.000000 0.000000 0.000000 0.182421 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.271382 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.271382 0.000000 0.000000 0.000000 0.238549 0.000000 0.000000
13 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.707107 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.707107 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
14 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.365175 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.365175 0.000000 0.000000 0.000000 0.000000 0.365175 0.000000 0.365175 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.402573 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.265333 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.245467 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.365175 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.201286 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
15 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.388607 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.341592 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.388607 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.388607 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.341592 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.341592 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.388607 0.214203 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
16 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.163762 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.129892 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.129892 0.163762 0.000000 0.000000 0.000000 0.000000 0.118988 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.143949 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.163762 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.163762 0.000000 0.000000 0.000000 0.163762 0.000000 0.000000 0.000000 0.000000 0.192043 0.000000 0.000000 0.163762 0.000000 0.000000 0.000000 0.000000 0.163762 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.090266 0.000000 0.000000 0.000000 0.000000 0.163762 0.143949 0.143949 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.143949 0.000000 0.143949 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.163762 0.000000 0.000000 0.129892 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.163762 0.090266 0.090266 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.163762 0.000000 0.000000 0.000000 0.000000 0.163762 0.000000 0.000000 0.000000 0.118988 0.163762 0.163762 0.129892 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.143949 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.143949 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.129892 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.163762 0.000000 0.163762 0.000000 0.000000 0.000000 0.000000 0.000000 0.163762 0.163762 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.110079 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.163762 0.000000 0.000000 0.000000 0.000000 0.143949 0.287898 0.000000 0.000000 0.163762 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.090266 0.000000 0.000000 0.143949 0.000000 0.000000 0.000000 0.000000 0.000000
17 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.505812 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.862644 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
18 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.122978 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.108100 0.000000 0.000000 0.000000 0.000000 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.000000 0.108100 0.122978 0.000000 0.122978 0.122978 0.122978 0.000000 0.000000 0.000000 0.122978 0.122978 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.097543 0.271145 0.000000 0.000000 0.000000 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.216325 0.135573 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.097543 0.000000 0.000000 0.000000 0.178710 0.122978 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.144217 0.000000 0.000000 0.271145 0.000000 0.000000 0.122978 0.000000 0.122978 0.000000 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.108100 0.000000 0.122978 0.122978 0.000000 0.216199 0.000000 0.000000 0.000000 0.108100 0.000000 0.000000 0.000000 0.122978 0.000000 0.000000 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.000000 0.247994 0.000000 0.000000 0.108100 0.000000 0.000000 0.245956 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.165329 0.000000 0.000000 0.195086 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.122978 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.245956 0.000000 0.000000 0.000000
19 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.504425 0.000000 0.000000 0.000000 0.000000 0.443397 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.400097 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.339070 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.443397 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.278042 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000

Dimensionality reduction

  • Massive matrices (ndocs×nvocabn_{docs} \times n_{vocab})

  • Even if sparse (contains mostly zeros), it might take up a lot of RAM to run algorithms on these

  • Plus, we often do not care about the words in a document but about the underlying meaning

  • Want vectors to be able to capture meaning

  • Reduce the dimensionality of the matrices to capture this meaning and lower computing load:

    • Apply Principal Component Analysis (PCA) to the matrix: called Latent Semantic Analysis
    • Latent Dirichlet Allocation (LDA)
    • Word embeddings: many approaches. We will discuss them a bit later.

Usefulness

  • After transformation, documents are represented as sequences of tokens or as vectors

  • Can now use these representations for our intended tasks

  • Compute document similarity, for instance by calculating cosine between two document-vectors (cosine-similarity)

  • Identify presence of concepts:

    • Count occurrences of words or dictionaries, build RegEx, train a ML algorithm, etc
    • Identify clusters of documents (ie of vectors)
    • Some of theses tasks only involve matrix products

Application of similarity analysis

  • Bertrand et al. (2021): Hall of Mirrors: Corporate Philanthropy and Strategic Advocacy

The paper in one line

  • Show that when nonprofits receive donations from firms, they tend to comment more on the same US federal regulatory rules but also adopt a closer type of comments
  • Approach to similarity analysis:

    1. Collapse the documents to organization-rule-year-level observations
    2. Apply LSA with tf-idf weighting to the matrix
    3. Compute cosine similarity between documents
    4. Regress similarity measure on dummy for donation (and FEs and controls)
  • Compare LSA to a Doc2vec and LDA on a similar task: predicting if two documents come from the same docket (documents pertaining to a same narrow topic)

On your own data set






On your own dataset

What similarity question could you ask on you own data?

Summary

NLP in Economics

  • Measuring document similarity
  • Concept detection
  • Relation between concepts
  • Associating text with metadata

Overal Approach

  • Get text data (ready-made, scrap, OCR, etc)
  • Pre-process the data
  • Transform data into useful format (a numeric array)
  • Run analysis. Several methods:
    • Dictionary-based
    • Rule-based
    • Machine Learning
    • Deep Learning
  • Use output in an econometric analysis

References

Ash, Elliott, Daniel L. Chen, and Arianna Ornaghi. 2024. “Gender Attitudes in the Judiciary: Evidence from US Circuit Courts.” American Economic Journal: Applied Economics 16 (1): 314–50. https://doi.org/10.1257/app.20210435.
Ash, Elliott, and Stephen Hansen. 2023. “Text Algorithms in Economics.” Annual Review of Economics 15 (1): 659–88. https://doi.org/10.1146/annurev-economics-082222-074352.
Bertrand, Marianne, Matilde Bombardini, Raymond Fisman, Brad Hackinen, and Francesco Trebbi. 2021. “Hall of Mirrors: Corporate Philanthropy and Strategic Advocacy.” The Quarterly Journal of Economics 136 (4): 2413–65. https://doi.org/10.1093/qje/qjab023.
Biasi, Barbara, and Song Ma. 2022. “The Education-Innovation Gap.” Working {{Paper}}. Working Paper Series. National Bureau of Economic Research. https://doi.org/10.3386/w29853.
Cagé, Julia, Nicolas Hervé, and Marie-Luce Viaud. 2020. “The Production of Information in an Online World.” The Review of Economic Studies 87 (5): 2126–64. https://doi.org/10.1093/restud/rdz061.
Djourelova, Milena, Ruben Durante, and Gregory J Martin. 2024. “The Impact of Online Competition on Local Newspapers: Evidence from the Introduction of Craigslist.” The Review of Economic Studies, May, rdae049. https://doi.org/10.1093/restud/rdae049.
Gentzkow, Matthew, Bryan Kelly, and Matt Taddy. 2019. “Text as Data.” Journal of Economic Literature 57 (3): 535–74. https://doi.org/10.1257/jel.20181020.
Gentzkow, Matthew, and Jesse M Shapiro. 2010. “What Drives Media Slant? Evidence From U.S. Daily Newspapers.” Econometrica 78 (1): 35–71. https://doi.org/10.3982/ECTA7195.
Kozlowski, Austin C., Matt Taddy, and James A. Evans. 2019. “The Geometry of Culture: Analyzing the Meanings of Class Through Word Embeddings.” American Sociological Review, September. https://doi.org/10.1177/0003122419877135.