diff --git a/data180recap/DATA180 Recap for DATA400.pdf b/data180recap/DATA180 Recap for DATA400.pdf new file mode 100644 index 0000000..c20ec68 Binary files /dev/null and b/data180recap/DATA180 Recap for DATA400.pdf differ diff --git a/ideas/AndrewScheiner-MiniProjectIdea1.md b/ideas/AndrewScheiner-MiniProjectIdea1.md new file mode 100644 index 0000000..23760a0 --- /dev/null +++ b/ideas/AndrewScheiner-MiniProjectIdea1.md @@ -0,0 +1,39 @@ +## Andrew Scheiner Mini Project Idea #1 +### Topic +The topic I am choosing to research for my mini-project is the [availability of computer science education](https://advocacy.code.org/report-data/) for middle school students across the United States. One reason why I am incredibly interested in this topic because for my educational research class, I wrote a research paper on the importance of increasing block-based coding usage in middle school classrooms. +### Collecting the Data +Code.org provides a [specific resource to view raw data](https://docs.google.com/spreadsheets/d/1TJ5-BmrTbcEcDDn3RSQccbLaAQ06t1YSsNxXGeBKOM0/edit?gid=0#gid=0) they collected in the form of Google sheets. This data includes multiple datapoints surrounding access of coding education seen across the country's middle schools. The Google sheet has data for: +- Access to CS education by state +- Student participation +- Participation by gender +- Participation by race/ethnicity +- Participation by other student groups +- General K-12 student demographics +Each aspect mentioned in this list has one tab within the available Google sheet. Each tab can be converted to its on CSV file for data analysis purposes. + +#### Data Source / Citation +- "Underlying data was contributed by various organizations, including the Computer Science Teachers Association (CSTA), University of Texas at Austin, the College Board, Technology Education and Literacy in Schools (TEALS), Project Lead the Way, BootUp PD, and many state education agencies." - [Code.org](https://advocacy.code.org/report-data/) +- Code.org. (2022). *Computer science access report data.* Retrieved from code.org/yourschool/accessreport. + +### Exploratory Data Analysis +#### Most Recent Access Data +| | Percent of Schools that Provided Data | Number of Middle School Offering CS | Total Number of Middle School | Percent of Middle School Offering CS | +|:------|----------------------------------------:|--------------------------------------:|--------------------------------:|---------------------------------------:| +| count | 52 | 52 | 52 | 52 | +| mean | 0.692091 | 468.865 | 1262.73 | 0.410832 | +| std | 0.241461 | 1671.11 | 4515.21 | 0.187131 | +| min | 0.143695 | 13 | 73 | 0.114247 | +| 25% | 0.506148 | 90.25 | 293.75 | 0.286079 | +| 50% | 0.680198 | 208.5 | 453 | 0.358464 | +| 75% | 0.915115 | 320.75 | 784.5 | 0.501176 | +| max | 1 | 12202 | 32891 | 0.918699 | + +### Model Idea +- For my project, I will most likely be generating visualizations based on data filters. I will accomplish this through one of two ways: for exploratory data analysis, I can create static graphs with data from each tab/aspect of the Google sheet, or I can create a dashboard on either Tableau or Streamlit so my audience can manually filter the middle school CS education data to see their own desired trends. However I decide to create my visuals, I should focus primarily on filterable choropleth maps since each row in my data represents a single US state. +- Additionally, I think I could use machine learning to discover patterns in the data which could be used to present findings and ideas to educators as well. I could regression models to explain the connection between access and demographics. The demographics would include state, gender, race/ethnicity, and student status. Student status includes learning disabilities and socio-economic disadvantages. Lastly, I could potentially use clustering to group states that have similar access qualities regarding middle school CS education. + +### Stakeholder Implications +My stakeholders (or audience) for this project would be middle school educators and those who oversee middle school cirriculum and success. The [Code.org dashboard](https://advocacy.code.org/report-data/) for viewing CS education data only allows viewers to filter and see data for high school classrooms. This project would be a great chance to make middle school data and trends more visually visible to educators. + +### Ethical Implications +The main ethical implications regarding my project are the issues with using, displaying, and analyzing educational data. Whenever working with educational data, it's crucial to ensure the raw data collected is without purposeful biases. Luckily, with my dataset, it is quantitative data where each instance corresponds to a singular US state. This removes the possibility of specific school districts, geographical areas, or educators/students being exposed for inequalities in my project. However, with the sections my data includes on gender, race/ethnicity, and other student demographics, which could have inherit but unintended biases on how CS educaton trends are distributed throughout the United States. The various organizations who contributed to the underlying data are mentioned below. \ No newline at end of file diff --git a/ideas/AndrewScheiner-MiniProjectIdea2.md b/ideas/AndrewScheiner-MiniProjectIdea2.md new file mode 100644 index 0000000..932af26 --- /dev/null +++ b/ideas/AndrewScheiner-MiniProjectIdea2.md @@ -0,0 +1,35 @@ +## Andrew Scheiner Mini Project Idea #2 +### Topic +The second idea I have for my mini project related to my discipline of educational studies is researching statistics on Teacher Shortage Areas (TSAs). These are specific aspects of schools that lack qualified teachers. I could create a lot of create research questions and visual elements with this topic. One idea I had was to create a interactive dashboard combining all of the subjects, states, and school years included in the school years I collect. + +### Collecting the Data, Data Source / Citation +My data source comes from the [U.S. Department of Education's TSA page](https://tsa.ed.gov/#/reports). This page lets users generate specific reports for TSA based on school year, state, and subject matter. For my idea, I have collected TSA data on all states and subjects from the 2024-25 school year. I was hoping to be able to collect data across multiple school years, however, the large number of instances seemed to freeze the TSA website when I would try and download data for more than several school years. For the project idea/proposal, I will just focus on 2024-25. + +### Exploratory Data Analysis +| State, Subject | Count | +|:-----------------------------------------------------------|----:| +| ('Kentucky', 'Science') | 29 | +| ('West Virginia', 'Language Arts') | 20 | +| ('West Virginia', 'Core Subjects') | 19 | +| ('West Virginia', 'Special Education') | 19 | +| ('West Virginia', 'Social Studies') | 19 | +| ('Kentucky', 'Mathematics') | 13 | +| ('Kentucky', 'Special Education') | 9 | +| ('West Virginia', 'Career and Technical Education') | 9 | +| ('West Virginia', 'Support Staff') | 9 | +| ('Ohio', 'Support Staff') | 7 | +| ('Kentucky', 'Language Arts') | 7 | +| ('Kentucky', 'Early Childhood') | 7 | +| ('Nevada', 'Special Education') | 7 | +| ('Puerto Rico', 'Special Education') | 6 | +| ('Washington', 'Support Staff') | 6 | +... + +### Model Idea +For a potential model to use for analyzing the TSA data, I would likely consult unsupervised learning methods such as clustering to determine groupings and similarities in areas across the country that are experiencing teacher shortages. We could also look at the state of shortages for specific subject areas and grades for different regions of the country and crosslist the findings with economic and population data. This could help give education officials, state officials, and researchers more information on what states have similar shortages and crosslisting with external data sources regarding money could help answer questions on funding. + +### Stakeholder Implications +My stakeholders (or audience) for this project would be government officials in the Department of Education who are looking to increase their knowledge/understanding of trends of TSAs across the United States. In addition, state officials who monitor education would be curious in my findings as well. This could help make policy decisions about necessary funding in different areas of the country. + +### Ethical Implications +The main ethical implication regarding my project is the quality and representation of the TSA entered into the government's database. It seems like the TSA data relies of user reporting - which begs the question of the quality and validity of the data. If the report input it vague and not interfaced well, this could decrease the dependability of using the data. When making decisions on TSA and funding policies across the country using this data source, it is critical that the most accurate information is portrayed, so the correct areas that need qualified educators are focused on. \ No newline at end of file diff --git a/ideas/MiddleSchoolData.xlsx b/ideas/MiddleSchoolData.xlsx new file mode 100644 index 0000000..27fe9ae Binary files /dev/null and b/ideas/MiddleSchoolData.xlsx differ diff --git a/ideas/TSAData_2024-25.xlsx b/ideas/TSAData_2024-25.xlsx new file mode 100644 index 0000000..d81ddfe Binary files /dev/null and b/ideas/TSAData_2024-25.xlsx differ diff --git a/ideas/eda_idea1.ipynb b/ideas/eda_idea1.ipynb new file mode 100644 index 0000000..d287b4e --- /dev/null +++ b/ideas/eda_idea1.ipynb @@ -0,0 +1,258 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Exploratory Data Analysis" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "from tabulate import tabulate" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['Most Recent Access Data', 'Most Recent Particiation Data', 'Gender Participation', 'RaceEthnicity', 'Other Student Groups', 'K-12 Student Demographics']\n" + ] + } + ], + "source": [ + "import pandas as pd\n", + "\n", + "# Load the entire workbook\n", + "excel_file_path = 'MiddleSchoolData.xlsx'\n", + "xl = pd.ExcelFile(excel_file_path)\n", + "\n", + "# List all sheet names\n", + "print(xl.sheet_names)\n", + "\n", + "# Read a specific sheet into a DataFrame\n", + "#df_sheet1 = pd.read_excel(excel_file_path, sheet_name='Sheet1')\n", + "\n", + "# If you want to read all sheets into a dictionary of DataFrames\n", + "dfs = {sheet_name: xl.parse(sheet_name) for sheet_name in xl.sheet_names}\n", + "\n", + "# Now dfs['Sheet1'], dfs['Sheet2'], etc., contain data from each sheet\n", + "#dfs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Most Recent Access Data" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "| | Percent of Schools that Provided Data | Number of Middle School Offering CS | Total Number of Middle School | Percent of Middle School Offering CS |\n", + "|:------|----------------------------------------:|--------------------------------------:|--------------------------------:|---------------------------------------:|\n", + "| count | 52 | 52 | 52 | 52 |\n", + "| mean | 0.692091 | 468.865 | 1262.73 | 0.410832 |\n", + "| std | 0.241461 | 1671.11 | 4515.21 | 0.187131 |\n", + "| min | 0.143695 | 13 | 73 | 0.114247 |\n", + "| 25% | 0.506148 | 90.25 | 293.75 | 0.286079 |\n", + "| 50% | 0.680198 | 208.5 | 453 | 0.358464 |\n", + "| 75% | 0.915115 | 320.75 | 784.5 | 0.501176 |\n", + "| max | 1 | 12202 | 32891 | 0.918699 |\n" + ] + } + ], + "source": [ + "describeAccess = dfs['Most Recent Access Data'].describe()\n", + "# Convert DataFrame to Markdown\n", + "markdown_table = tabulate(describeAccess, headers='keys', tablefmt='pipe')\n", + "\n", + "print(markdown_table)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Most Recent Particiation Data" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Unnamed: 0Number of Middle School Female Students Enrolled in Computer SciencePercent of Middle School Female Students Enrolled in Computer ScienceNumber of Middle School Male Students Enrolled in Computer SciencePercent of Middle School Male Students Enrolled in Computer ScienceNumber of Middle School Non-Binary Students Enrolled in Computer SciencePercent of Middle School Non-Binary Students Enrolled in Computer Science
count52525252525252
unique5234313431105
topNational----00
freq1191919192526
\n", + "
" + ], + "text/plain": [ + " Unnamed: 0 \\\n", + "count 52 \n", + "unique 52 \n", + "top National \n", + "freq 1 \n", + "\n", + " Number of Middle School Female Students Enrolled in Computer Science \\\n", + "count 52 \n", + "unique 34 \n", + "top - \n", + "freq 19 \n", + "\n", + " Percent of Middle School Female Students Enrolled in Computer Science \\\n", + "count 52 \n", + "unique 31 \n", + "top - \n", + "freq 19 \n", + "\n", + " Number of Middle School Male Students Enrolled in Computer Science \\\n", + "count 52 \n", + "unique 34 \n", + "top - \n", + "freq 19 \n", + "\n", + " Percent of Middle School Male Students Enrolled in Computer Science \\\n", + "count 52 \n", + "unique 31 \n", + "top - \n", + "freq 19 \n", + "\n", + " Number of Middle School Non-Binary Students Enrolled in Computer Science \\\n", + "count 52 \n", + "unique 10 \n", + "top 0 \n", + "freq 25 \n", + "\n", + " Percent of Middle School Non-Binary Students Enrolled in Computer Science \n", + "count 52 \n", + "unique 5 \n", + "top 0 \n", + "freq 26 " + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dfs['Gender Participation'].describe()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.13" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/ideas/eda_idea2.ipynb b/ideas/eda_idea2.ipynb new file mode 100644 index 0000000..06e534f --- /dev/null +++ b/ideas/eda_idea2.ipynb @@ -0,0 +1,839 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Exploratory Data Analysis" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from tabulate import tabulate\n", + "import pandas as pd" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
School YearStateCountrySchool DistrictGeographic RegionSubject MatterDisciplineGradesGrades by Age
02024_2025Alabama---MathematicsBasic and Advanced Mathematics6, 7, 8, 9, 10, 11, 12-
12024_2025Alabama---ScienceGeneral Science6, 7, 8, 9, 10, 11, 12Age above 5
22024_2025Alabama---Special EducationAll ExceptionalitiesPre-K, K, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12-
32024_2025Alabama---Core SubjectsElementary EducationK, 1, 2, 3, 4, 5, 6-
42024_2025Alaska---Special EducationAll ExceptionalitiesPre-K, K, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12-
\n", + "
" + ], + "text/plain": [ + " School Year State Country School District Geographic Region \\\n", + "0 2024_2025 Alabama - - - \n", + "1 2024_2025 Alabama - - - \n", + "2 2024_2025 Alabama - - - \n", + "3 2024_2025 Alabama - - - \n", + "4 2024_2025 Alaska - - - \n", + "\n", + " Subject Matter Discipline \\\n", + "0 Mathematics Basic and Advanced Mathematics \n", + "1 Science General Science \n", + "2 Special Education All Exceptionalities \n", + "3 Core Subjects Elementary Education \n", + "4 Special Education All Exceptionalities \n", + "\n", + " Grades Grades by Age \n", + "0 6, 7, 8, 9, 10, 11, 12 - \n", + "1 6, 7, 8, 9, 10, 11, 12 Age above 5 \n", + "2 Pre-K, K, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 - \n", + "3 K, 1, 2, 3, 4, 5, 6 - \n", + "4 Pre-K, K, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 - " + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Read a specific sheet into a DataFrame\n", + "dataTSA = pd.read_excel('TSAData_2024-25.xlsx', sheet_name='TSA')\n", + "dataTSA.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "| | 0 |\n", + "|:-----------------------------------------------------------|----:|\n", + "| ('Kentucky', 'Science') | 29 |\n", + "| ('West Virginia', 'Language Arts') | 20 |\n", + "| ('West Virginia', 'Core Subjects') | 19 |\n", + "| ('West Virginia', 'Special Education') | 19 |\n", + "| ('West Virginia', 'Social Studies') | 19 |\n", + "| ('Kentucky', 'Mathematics') | 13 |\n", + "| ('Kentucky', 'Special Education') | 9 |\n", + "| ('West Virginia', 'Career and Technical Education') | 9 |\n", + "| ('West Virginia', 'Support Staff') | 9 |\n", + "| ('Ohio', 'Support Staff') | 7 |\n", + "| ('Kentucky', 'Language Arts') | 7 |\n", + "| ('Kentucky', 'Early Childhood') | 7 |\n", + "| ('Nevada', 'Special Education') | 7 |\n", + "| ('Puerto Rico', 'Special Education') | 6 |\n", + "| ('Washington', 'Support Staff') | 6 |\n", + "| ('Maryland', 'World Languages') | 6 |\n", + "| ('Maryland', 'Science') | 5 |\n", + "| ('Puerto Rico', 'Science') | 5 |\n", + "| ('Maryland', 'Special Education') | 5 |\n", + "| ('Minnesota', 'Special Education') | 5 |\n", + "| ('South Carolina', 'Career and Technical Education') | 5 |\n", + "| ('Iowa', 'Career and Technical Education') | 5 |\n", + "| ('Kansas', 'Special Education') | 5 |\n", + "| ('West Virginia', 'Mathematics') | 5 |\n", + "| ('West Virginia', 'Science') | 5 |\n", + "| ('Iowa', 'Science') | 5 |\n", + "| ('Nebraska', 'Support Staff') | 4 |\n", + "| ('Kentucky', 'World Languages') | 4 |\n", + "| ('West Virginia', 'Health and Physical Fitness') | 4 |\n", + "| ('Maryland', 'Art and Music Education') | 4 |\n", + "| ('Maryland', 'Career and Technical Education') | 4 |\n", + "| ('Rhode Island', 'Science') | 4 |\n", + "| ('Rhode Island', 'Special Education') | 4 |\n", + "| ('South Carolina', 'Art and Music Education') | 4 |\n", + "| ('Colorado', 'Career and Technical Education') | 4 |\n", + "| ('South Carolina', 'Support Staff') | 4 |\n", + "| ('Wyoming', 'Science') | 3 |\n", + "| ('Michigan', 'Support Staff') | 3 |\n", + "| ('Nevada', 'Science') | 3 |\n", + "| ('Nevada', 'Support Staff') | 3 |\n", + "| ('Delaware', 'Special Education') | 3 |\n", + "| ('Kentucky', 'English as a Second Language') | 3 |\n", + "| ('Delaware', 'Career and Technical Education') | 3 |\n", + "| ('Maine', 'Special Education') | 3 |\n", + "| ('South Carolina', 'Core Subjects') | 3 |\n", + "| ('North Carolina', 'Special Education') | 3 |\n", + "| ('Hawaii', 'World Languages') | 3 |\n", + "| ('Arizona', 'Special Education') | 3 |\n", + "| ('Colorado', 'Support Staff') | 3 |\n", + "| ('Michigan', 'Career and Technical Education') | 3 |\n", + "| ('Maine', 'Art and Music Education') | 2 |\n", + "| ('Maryland', 'Health and Physical Fitness') | 2 |\n", + "| ('Iowa', 'Special Education') | 2 |\n", + "| ('Iowa', 'Support Staff') | 2 |\n", + "| ('Maryland', 'Social Studies') | 2 |\n", + "| ('Iowa', 'Language Arts') | 2 |\n", + "| ('Iowa', 'Health and Physical Fitness') | 2 |\n", + "| ('Michigan', 'Health and Physical Fitness') | 2 |\n", + "| ('Nebraska', 'Art and Music Education') | 2 |\n", + "| ('Michigan', 'Language Arts') | 2 |\n", + "| ('Michigan', 'Special Education') | 2 |\n", + "| ('Wisconsin', 'English as a Second Language') | 2 |\n", + "| ('Wisconsin', 'Core Subjects') | 2 |\n", + "| ('West Virginia', 'World Languages') | 2 |\n", + "| ('West Virginia', 'Art and Music Education') | 2 |\n", + "| ('Washington', 'Career and Technical Education') | 2 |\n", + "| ('Vermont', 'Support Staff') | 2 |\n", + "| ('South Carolina', 'Language Arts') | 2 |\n", + "| ('South Carolina', 'Health and Physical Fitness') | 2 |\n", + "| ('Rhode Island', 'Mathematics') | 2 |\n", + "| ('Rhode Island', 'English as a Second Language') | 2 |\n", + "| ('Puerto Rico', 'Support Staff') | 2 |\n", + "| ('Ohio', 'Career and Technical Education') | 2 |\n", + "| ('Ohio', 'Art and Music Education') | 2 |\n", + "| ('North Carolina', 'Social Studies') | 2 |\n", + "| ('North Carolina', 'Science') | 2 |\n", + "| ('Iowa', 'Art and Music Education') | 2 |\n", + "| ('North Carolina', 'Language Arts') | 2 |\n", + "| ('New Jersey', 'English as a Second Language') | 2 |\n", + "| ('Nevada', 'Health and Physical Fitness') | 2 |\n", + "| ('Nevada', 'Core Subjects') | 2 |\n", + "| ('Montana', 'Support Staff') | 2 |\n", + "| ('Montana', 'Art and Music Education') | 2 |\n", + "| ('Missouri', 'Special Education') | 2 |\n", + "| ('North Carolina', 'Mathematics') | 2 |\n", + "| ('Wyoming', 'World Languages') | 2 |\n", + "| ('American Samoa', 'Language Arts') | 2 |\n", + "| ('Arizona', 'Science') | 2 |\n", + "| ('Connecticut', 'Support Staff') | 2 |\n", + "| ('Connecticut', 'English as a Second Language') | 2 |\n", + "| ('American Samoa', 'Science') | 2 |\n", + "| ('American Samoa', 'Social Studies') | 2 |\n", + "| ('Florida', 'Science') | 2 |\n", + "| ('Connecticut', 'Special Education') | 2 |\n", + "| ('American Samoa', 'Mathematics') | 2 |\n", + "| ('Colorado', 'Art and Music Education') | 1 |\n", + "| ('Colorado', 'Core Subjects') | 1 |\n", + "| ('South Carolina', 'Academically Advanced') | 1 |\n", + "| ('Illinois', 'Social Studies') | 1 |\n", + "| ('Rhode Island', 'World Languages') | 1 |\n", + "| ('Colorado', 'Early Childhood') | 1 |\n", + "| ('Colorado', 'Health and Physical Fitness') | 1 |\n", + "| ('Colorado', 'Language Arts') | 1 |\n", + "| ('Rhode Island', 'Language Arts') | 1 |\n", + "| ('Rhode Island', 'Health and Physical Fitness') | 1 |\n", + "| ('Rhode Island', 'General Shortages') | 1 |\n", + "| ('Rhode Island', 'Social Studies') | 1 |\n", + "| ('South Carolina', 'Computer Science') | 1 |\n", + "| ('Rhode Island', 'Art and Music Education') | 1 |\n", + "| ('California', 'Language Arts') | 1 |\n", + "| ('South Carolina', 'Drivers Education') | 1 |\n", + "| ('South Carolina', 'English as a Second Language') | 1 |\n", + "| ('California', 'Core Subjects') | 1 |\n", + "| ('Arkansas', 'World Languages') | 1 |\n", + "| ('South Carolina', 'Mathematics') | 1 |\n", + "| ('South Carolina', 'Science') | 1 |\n", + "| ('South Carolina', 'Social Studies') | 1 |\n", + "| ('South Carolina', 'Special Education') | 1 |\n", + "| ('Arkansas', 'Social Studies') | 1 |\n", + "| ('South Dakota', 'Career and Technical Education') | 1 |\n", + "| ('South Dakota', 'Language Arts') | 1 |\n", + "| ('South Dakota', 'Science') | 1 |\n", + "| ('South Dakota', 'Special Education') | 1 |\n", + "| ('Colorado', 'Social Studies') | 1 |\n", + "| ('Connecticut', 'Mathematics') | 1 |\n", + "| ('Connecticut', 'Career and Technical Education') | 1 |\n", + "| ('Oregon', 'Core Subjects') | 1 |\n", + "| ('North Dakota', 'Art and Music Education') | 1 |\n", + "| ('North Dakota', 'Career and Technical Education') | 1 |\n", + "| ('North Dakota', 'General Shortages') | 1 |\n", + "| ('North Dakota', 'Special Education') | 1 |\n", + "| ('North Dakota', 'Support Staff') | 1 |\n", + "| ('Connecticut', 'Social Studies') | 1 |\n", + "| ('Ohio', 'Computer Science') | 1 |\n", + "| ('Ohio', 'Special Education') | 1 |\n", + "| ('Connecticut', 'Science') | 1 |\n", + "| ('Oklahoma', 'Early Childhood') | 1 |\n", + "| ('Oklahoma', 'Language Arts') | 1 |\n", + "| ('Oklahoma', 'Social Studies') | 1 |\n", + "| ('Oregon', 'Language Arts') | 1 |\n", + "| ('Puerto Rico', 'Social Studies') | 1 |\n", + "| ('Oregon', 'Mathematics') | 1 |\n", + "| ('Oregon', 'Science') | 1 |\n", + "| ('Oregon', 'Special Education') | 1 |\n", + "| ('Pennsylvania', 'Career and Technical Education') | 1 |\n", + "| ('Pennsylvania', 'Core Subjects') | 1 |\n", + "| ('Pennsylvania', 'Health and Physical Fitness') | 1 |\n", + "| ('Pennsylvania', 'Mathematics') | 1 |\n", + "| ('Pennsylvania', 'Science') | 1 |\n", + "| ('Pennsylvania', 'Special Education') | 1 |\n", + "| ('Puerto Rico', 'English as a Second Language') | 1 |\n", + "| ('Puerto Rico', 'Mathematics') | 1 |\n", + "| ('Tennessee', 'English as a Second Language') | 1 |\n", + "| ('Tennessee', 'Early Childhood') | 1 |\n", + "| ('Texas', 'Career and Technical Education') | 1 |\n", + "| ('Tennessee', 'Special Education') | 1 |\n", + "| ('Washington', 'Language Arts') | 1 |\n", + "| ('Arkansas', 'Language Arts') | 1 |\n", + "| ('Washington', 'World Languages') | 1 |\n", + "| ('Arkansas', 'Computer Science') | 1 |\n", + "| ('Arkansas', 'Art and Music Education') | 1 |\n", + "| ('Arizona', 'Social Studies') | 1 |\n", + "| ('West Virginia', 'Early Childhood') | 1 |\n", + "| ('Arizona', 'Mathematics') | 1 |\n", + "| ('Arizona', 'Language Arts') | 1 |\n", + "| ('American Samoa', 'Special Education') | 1 |\n", + "| ('Wisconsin', 'Art and Music Education') | 1 |\n", + "| ('Alaska', 'Special Education') | 1 |\n", + "| ('Alabama', 'Special Education') | 1 |\n", + "| ('Wisconsin', 'Language Arts') | 1 |\n", + "| ('Wisconsin', 'Mathematics') | 1 |\n", + "| ('Wisconsin', 'Psychology') | 1 |\n", + "| ('Wisconsin', 'Special Education') | 1 |\n", + "| ('Wisconsin', 'Support Staff') | 1 |\n", + "| ('Wyoming', 'Academic Intervention') | 1 |\n", + "| ('Wyoming', 'Art and Music Education') | 1 |\n", + "| ('Wyoming', 'Computer Science') | 1 |\n", + "| ('Wyoming', 'English as a Second Language') | 1 |\n", + "| ('Wyoming', 'General Shortages') | 1 |\n", + "| ('Wyoming', 'Health and Physical Fitness') | 1 |\n", + "| ('Wyoming', 'Language Arts') | 1 |\n", + "| ('Alabama', 'Science') | 1 |\n", + "| ('Wyoming', 'Special Education') | 1 |\n", + "| ('Wyoming', 'Support Staff') | 1 |\n", + "| ('Washington', 'Mathematics') | 1 |\n", + "| ('Washington', 'English as a Second Language') | 1 |\n", + "| ('Tennessee', 'World Languages') | 1 |\n", + "| ('Washington', 'Early Childhood') | 1 |\n", + "| ('Delaware', 'English as a Second Language') | 1 |\n", + "| ('Texas', 'Computer Science') | 1 |\n", + "| ('Texas', 'English as a Second Language') | 1 |\n", + "| ('Texas', 'Special Education') | 1 |\n", + "| ('U.S. Virgin Islands', 'Career and Technical Education') | 1 |\n", + "| ('U.S. Virgin Islands', 'Core Subjects') | 1 |\n", + "| ('U.S. Virgin Islands', 'Health and Physical Fitness') | 1 |\n", + "| ('U.S. Virgin Islands', 'Language Arts') | 1 |\n", + "| ('U.S. Virgin Islands', 'Mathematics') | 1 |\n", + "| ('Utah', 'Core Subjects') | 1 |\n", + "| ('Vermont', 'Core Subjects') | 1 |\n", + "| ('Vermont', 'Health and Physical Fitness') | 1 |\n", + "| ('Vermont', 'Language Arts') | 1 |\n", + "| ('Arkansas', 'Science') | 1 |\n", + "| ('Virginia', 'Career and Technical Education') | 1 |\n", + "| ('Virginia', 'Core Subjects') | 1 |\n", + "| ('Virginia', 'General Shortages') | 1 |\n", + "| ('Virginia', 'Language Arts') | 1 |\n", + "| ('Virginia', 'Mathematics') | 1 |\n", + "| ('Virginia', 'Science') | 1 |\n", + "| ('Virginia', 'Social Studies') | 1 |\n", + "| ('Virginia', 'Special Education') | 1 |\n", + "| ('Virginia', 'Support Staff') | 1 |\n", + "| ('Virginia', 'World Languages') | 1 |\n", + "| ('Arkansas', 'Mathematics') | 1 |\n", + "| ('Washington', 'Computer Science') | 1 |\n", + "| ('Washington', 'Core Subjects') | 1 |\n", + "| ('Connecticut', 'World Languages') | 1 |\n", + "| ('Delaware', 'Language Arts') | 1 |\n", + "| ('Illinois', 'Special Education') | 1 |\n", + "| ('Delaware', 'Mathematics') | 1 |\n", + "| ('Hawaii', 'Mathematics') | 1 |\n", + "| ('Maine', 'Computer Science') | 1 |\n", + "| ('Maine', 'Core Subjects') | 1 |\n", + "| ('Maine', 'Early Childhood') | 1 |\n", + "| ('Maine', 'Health and Physical Fitness') | 1 |\n", + "| ('Maine', 'Language Arts') | 1 |\n", + "| ('Maine', 'Social Studies') | 1 |\n", + "| ('Hawaii', 'Language Arts') | 1 |\n", + "| ('Maryland', 'Academic Intervention') | 1 |\n", + "| ('Hawaii', 'English as a Second Language') | 1 |\n", + "| ('Hawaii', 'Career and Technical Education') | 1 |\n", + "| ('Maryland', 'Computer Science') | 1 |\n", + "| ('Maryland', 'Core Subjects') | 1 |\n", + "| ('Maryland', 'Early Childhood') | 1 |\n", + "| ('Maryland', 'English as a Second Language') | 1 |\n", + "| ('Georgia', 'Special Education') | 1 |\n", + "| ('Maryland', 'Language Arts') | 1 |\n", + "| ('Maryland', 'Mathematics') | 1 |\n", + "| ('Georgia', 'Science') | 1 |\n", + "| ('Florida', 'Special Education') | 1 |\n", + "| ('Florida', 'Mathematics') | 1 |\n", + "| ('Massachusetts', 'Computer Science') | 1 |\n", + "| ('Michigan', 'Art and Music Education') | 1 |\n", + "| ('Florida', 'Language Arts') | 1 |\n", + "| ('Michigan', 'Core Subjects') | 1 |\n", + "| ('Michigan', 'English as a Second Language') | 1 |\n", + "| ('Florida', 'English as a Second Language') | 1 |\n", + "| ('District of Columbia', 'World Languages') | 1 |\n", + "| ('Michigan', 'Mathematics') | 1 |\n", + "| ('Louisiana', 'English as a Second Language') | 1 |\n", + "| ('Hawaii', 'Science') | 1 |\n", + "| ('Hawaii', 'Special Education') | 1 |\n", + "| ('Iowa', 'Mathematics') | 1 |\n", + "| ('Illinois', 'Support Staff') | 1 |\n", + "| ('Illinois', 'World Languages') | 1 |\n", + "| ('Indiana', 'Mathematics') | 1 |\n", + "| ('Iowa', 'Academic Intervention') | 1 |\n", + "| ('Iowa', 'Academically Advanced') | 1 |\n", + "| ('Illinois', 'Science') | 1 |\n", + "| ('Illinois', 'Mathematics') | 1 |\n", + "| ('Iowa', 'Computer Science') | 1 |\n", + "| ('Iowa', 'Core Subjects') | 1 |\n", + "| ('Iowa', 'Early Childhood') | 1 |\n", + "| ('Iowa', 'English as a Second Language') | 1 |\n", + "| ('Illinois', 'Language Arts') | 1 |\n", + "| ('Illinois', 'Health and Physical Fitness') | 1 |\n", + "| ('Illinois', 'General Shortages') | 1 |\n", + "| ('Kentucky', 'Social Studies') | 1 |\n", + "| ('Iowa', 'Social Studies') | 1 |\n", + "| ('Illinois', 'English as a Second Language') | 1 |\n", + "| ('Illinois', 'Career and Technical Education') | 1 |\n", + "| ('Iowa', 'World Languages') | 1 |\n", + "| ('Kansas', 'Academically Advanced') | 1 |\n", + "| ('Illinois', 'Art and Music Education') | 1 |\n", + "| ('Kentucky', 'Academically Advanced') | 1 |\n", + "| ('Kentucky', 'Career and Technical Education') | 1 |\n", + "| ('Illinois', 'Administrative') | 1 |\n", + "| ('Idaho', 'Mathematics') | 1 |\n", + "| ('Idaho', 'Language Arts') | 1 |\n", + "| ('Idaho', 'Health and Physical Fitness') | 1 |\n", + "| ('Idaho', 'Core Subjects') | 1 |\n", + "| ('Michigan', 'Psychology') | 1 |\n", + "| ('Michigan', 'Science') | 1 |\n", + "| ('Michigan', 'Social Studies') | 1 |\n", + "| ('New Jersey', 'Special Education') | 1 |\n", + "| ('District of Columbia', 'Early Childhood') | 1 |\n", + "| ('Nevada', 'Early Childhood') | 1 |\n", + "| ('District of Columbia', 'Core Subjects') | 1 |\n", + "| ('Nevada', 'Language Arts') | 1 |\n", + "| ('Nevada', 'Mathematics') | 1 |\n", + "| ('District of Columbia', 'Career and Technical Education') | 1 |\n", + "| ('Nevada', 'Social Studies') | 1 |\n", + "| ('District of Columbia', 'Art and Music Education') | 1 |\n", + "| ('Delaware', 'World Languages') | 1 |\n", + "| ('New Jersey', 'Career and Technical Education') | 1 |\n", + "| ('Delaware', 'Science') | 1 |\n", + "| ('New Jersey', 'Mathematics') | 1 |\n", + "| ('New Jersey', 'Science') | 1 |\n", + "| ('New Jersey', 'World Languages') | 1 |\n", + "| ('Nebraska', 'World Languages') | 1 |\n", + "| ('New Mexico', 'Core Subjects') | 1 |\n", + "| ('New Mexico', 'Early Childhood') | 1 |\n", + "| ('New Mexico', 'English as a Second Language') | 1 |\n", + "| ('New Mexico', 'Mathematics') | 1 |\n", + "| ('New Mexico', 'Science') | 1 |\n", + "| ('New Mexico', 'Special Education') | 1 |\n", + "| ('New York', 'Career and Technical Education') | 1 |\n", + "| ('New York', 'Health and Physical Fitness') | 1 |\n", + "| ('New York', 'Language Arts') | 1 |\n", + "| ('New York', 'Science') | 1 |\n", + "| ('New York', 'Support Staff') | 1 |\n", + "| ('North Carolina', 'Career and Technical Education') | 1 |\n", + "| ('North Carolina', 'Core Subjects') | 1 |\n", + "| ('Nevada', 'Computer Science') | 1 |\n", + "| ('District of Columbia', 'English as a Second Language') | 1 |\n", + "| ('District of Columbia', 'Special Education') | 1 |\n", + "| ('Montana', 'Science') | 1 |\n", + "| ('District of Columbia', 'Social Studies') | 1 |\n", + "| ('Michigan', 'World Languages') | 1 |\n", + "| ('District of Columbia', 'Science') | 1 |\n", + "| ('Mississippi', 'Mathematics') | 1 |\n", + "| ('Mississippi', 'Science') | 1 |\n", + "| ('Mississippi', 'Special Education') | 1 |\n", + "| ('Mississippi', 'World Languages') | 1 |\n", + "| ('Missouri', 'Early Childhood') | 1 |\n", + "| ('District of Columbia', 'Mathematics') | 1 |\n", + "| ('District of Columbia', 'Language Arts') | 1 |\n", + "| ('Montana', 'Career and Technical Education') | 1 |\n", + "| ('Montana', 'Language Arts') | 1 |\n", + "| ('Montana', 'Mathematics') | 1 |\n", + "| ('Montana', 'Social Studies') | 1 |\n", + "| ('Nebraska', 'Special Education') | 1 |\n", + "| ('Montana', 'Special Education') | 1 |\n", + "| ('District of Columbia', 'Health and Physical Fitness') | 1 |\n", + "| ('Montana', 'World Languages') | 1 |\n", + "| ('Alabama', 'Mathematics') | 1 |\n", + "| ('Nebraska', 'Career and Technical Education') | 1 |\n", + "| ('Nebraska', 'Core Subjects') | 1 |\n", + "| ('Nebraska', 'Early Childhood') | 1 |\n", + "| ('Nebraska', 'English as a Second Language') | 1 |\n", + "| ('Nebraska', 'Health and Physical Fitness') | 1 |\n", + "| ('Nebraska', 'Language Arts') | 1 |\n", + "| ('Nebraska', 'Mathematics') | 1 |\n", + "| ('Nebraska', 'Science') | 1 |\n", + "| ('Nebraska', 'Social Studies') | 1 |\n", + "| ('Alabama', 'Core Subjects') | 1 |\n" + ] + } + ], + "source": [ + "# Get value counts\n", + "# Convert DataFrame to Markdown\n", + "print(tabulate(pd.DataFrame(dataTSA[['State', 'Subject Matter']].value_counts()),\\\n", + " headers='keys', tablefmt='pipe'))" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
School YearCountrySchool DistrictGeographic RegionDisciplineGradesGrades by Age
Subject MatterState
Academic InterventionIowa1111111
Maryland1111111
Wyoming1111111
Academically AdvancedIowa1111111
Kansas1111111
Kentucky1111111
South Carolina1111111
AdministrativeIllinois1111111
Art and Music EducationArkansas1111111
Colorado1111111
District of Columbia1111111
Illinois1111111
Iowa2222222
Maine2222222
Maryland4444444
\n", + "
" + ], + "text/plain": [ + " School Year Country \\\n", + "Subject Matter State \n", + "Academic Intervention Iowa 1 1 \n", + " Maryland 1 1 \n", + " Wyoming 1 1 \n", + "Academically Advanced Iowa 1 1 \n", + " Kansas 1 1 \n", + " Kentucky 1 1 \n", + " South Carolina 1 1 \n", + "Administrative Illinois 1 1 \n", + "Art and Music Education Arkansas 1 1 \n", + " Colorado 1 1 \n", + " District of Columbia 1 1 \n", + " Illinois 1 1 \n", + " Iowa 2 2 \n", + " Maine 2 2 \n", + " Maryland 4 4 \n", + "\n", + " School District \\\n", + "Subject Matter State \n", + "Academic Intervention Iowa 1 \n", + " Maryland 1 \n", + " Wyoming 1 \n", + "Academically Advanced Iowa 1 \n", + " Kansas 1 \n", + " Kentucky 1 \n", + " South Carolina 1 \n", + "Administrative Illinois 1 \n", + "Art and Music Education Arkansas 1 \n", + " Colorado 1 \n", + " District of Columbia 1 \n", + " Illinois 1 \n", + " Iowa 2 \n", + " Maine 2 \n", + " Maryland 4 \n", + "\n", + " Geographic Region Discipline \\\n", + "Subject Matter State \n", + "Academic Intervention Iowa 1 1 \n", + " Maryland 1 1 \n", + " Wyoming 1 1 \n", + "Academically Advanced Iowa 1 1 \n", + " Kansas 1 1 \n", + " Kentucky 1 1 \n", + " South Carolina 1 1 \n", + "Administrative Illinois 1 1 \n", + "Art and Music Education Arkansas 1 1 \n", + " Colorado 1 1 \n", + " District of Columbia 1 1 \n", + " Illinois 1 1 \n", + " Iowa 2 2 \n", + " Maine 2 2 \n", + " Maryland 4 4 \n", + "\n", + " Grades Grades by Age \n", + "Subject Matter State \n", + "Academic Intervention Iowa 1 1 \n", + " Maryland 1 1 \n", + " Wyoming 1 1 \n", + "Academically Advanced Iowa 1 1 \n", + " Kansas 1 1 \n", + " Kentucky 1 1 \n", + " South Carolina 1 1 \n", + "Administrative Illinois 1 1 \n", + "Art and Music Education Arkansas 1 1 \n", + " Colorado 1 1 \n", + " District of Columbia 1 1 \n", + " Illinois 1 1 \n", + " Iowa 2 2 \n", + " Maine 2 2 \n", + " Maryland 4 4 " + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Get summary by grouping\n", + "dataTSA.groupby(['Subject Matter', 'State']).count().head(15)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.13" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}