Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .DS_Store
Binary file not shown.
Binary file added Ideas/.DS_Store
Binary file not shown.
17 changes: 17 additions & 0 deletions Ideas/Mini Project #1 Idea.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# DATA 400 - Ashley Kim
### Mini Project #1
##### Research Question: How Does the Location of Universities Affect Their Racial Diversity?

This project aims to explore the relationship between university location and racial diversity on both the institutional level and degree level. The focus will be on Bachelor’s degree awarded each year at institutions in the United States. This research has meaningful implications for universities, policymakers, and students. Universities can use the findings to refine recruitment and retention strategies, particularly in underrepresented areas. Policymakers can leverage insights to shape education policies that promote equitable access to higher education, ensuring diversity across institutions regardless of location. Prospective students may benefit from understanding how institutional location affects racial representation in different academic fields. If the results indicate disparities in degree attainment by race in certain geographic settings, interventions and efforts can be made to promote inclusivity. Also, this project contributes to broader societal discussions on equity in higher education by examining how geographic factors influence racial diversity. The findings could inform efforts to create more inclusive and accessible academic environments across all types of institutions.

### Data Source
I will use publicly available datasets from the [Integrated Postsecondary Education Data System] (IPEDS). This comprehensive and federally maintained source collects standardized data on postsecondary institutions across the United States. IPEDS provides annual reports on key metrics such as institutional characteristics, enrollment demographics, and degree completions, ensuring data consistency and reliability. The data retrieval process will involve downloading “Institutional Characteristics” and “Completions” datasets from IPEDS. I will focus on extracting variables such as racial demographics, bachelor’s degree awarded by major and race, geographic location of universities, locale codes, and institutional characters such as institution size and institution type.


### Model Specification
This project will use a multivariate regression model to analyze the relationship between university location and racial diversity in degree attainment. The dependent variable will be the proportion of Bachelor's degrees awarded to different racial groups across majors and institutions. Independent variables will include locale codes to capture geographic settings, while control variables will account for institutional characteristics, such as size and type, to ensure robustness. This model is well-suited for identifying patterns and potential disparities across different regions, though it does not establish causal relationships.



[//]: #
[integrated postsecondary education data system]: <https://nces.ed.gov/ipeds/datacenter/DataFiles.aspx?gotoReportId=7&fromIpeds=true&sid=37c646f5-c31a-4355-a887-a92d2c297362&rtid=7>
19 changes: 19 additions & 0 deletions Ideas/Mini Project #2 Idea.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# DATA 400 - Ashley Kim
### Mini Project #2
##### Research Question: How Does Payroll Affect Team Performance in Major League Baseball?


It started with a question I had: “Do richer teams (that can pay for more "valuable players") perform better?” Unlike other sports and leagues, Major League Baseball (MLB) does not have a salary cap, meaning teams can spend as much as their budget allows when building their rosters. Given the absence of a salary cap in MLB, this research investigates whether higher payrolls are associated with improved team performance, as measured by win percentage. This project aims to explore the relationship between team payroll and performance in MLB over the past decade (2014–2024). If a relationship is observed, the findings may have various implications for stakeholders. Team owners and executives might consider how payroll investments are linked to team performance when making financial decisions related to player acquisitions. For players and agents, understanding the potential connection between payroll and team success could influence how they approach contract negotiations and value players' contributions. Additionally, if significant disparities in payroll exist between teams, this could raise discussions about competitive fairness and its impact on the overall league structure.


### Data Source
I will use team payroll data for MLB teams over the past 10 years (2014–2024) from [Spotrac]. The data encompasses player salaries, bonuses, and incentives, though it is not directly downloadable. To retrieve it, I will use BeautifulSoup to scrape data from Spotrac. For performance metrics, I will download team statistics data from [Baseball-Reference] for the same period. Key variables will include wins, Runs Scored per Game (RS), Runs Allowed per Game (RA), and On-base Plus Slugging (OPS). After gathering both payroll and performance data, I will structure it into a panel format, which will combine cross-sectional data for each team over multiple years.


### Model Specification
This project will use a panel regression model to analyze the relationship between team payroll and performance in MLB. The dependent variable will be team win percentage, calculated as the ratio of wins to games played. The primary independent variable will be team payroll, including salaries, bonuses, and player incentives. Control variables will include Runs Scored per Game (RS) and Runs Allowed per Game (RA) to account for offensive and defensive performance, as well as On-base Plus Slugging (OPS) to capture overall offensive strength. The model will also include the lagged win percentage to control for carryover effects from the previous season. To ensure the reliability of the results, I will check for potential multicollinearity to minimize distortion of coefficient estimates due to high correlations among the predictors. This model is well-suited for identifying patterns in the relationship between payroll and performance while adjusting for key performance metrics, though it does not establish causal relationships.


[//]: #
[spotrac]:<https://www.spotrac.com/mlb/payroll/_/year/2024/sort/cap_total2>
[baseball-reference]: <https://www.baseball-reference.com/leagues/majors/2024.shtml>
Binary file not shown.
Binary file not shown.
Binary file added NYC Project Final Presentation.pptx
Binary file not shown.
Binary file added notes/.DS_Store
Binary file not shown.
92 changes: 92 additions & 0 deletions nyc_taxi_tip_prediction_readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
## NYC Taxi Tip Prediction Based on Neighborhood Median Income

### Ashley Kim & John Park

![](/Users/yelim/Downloads/big-city-1265055_1280.jpg){width="1000"}

## Description

Tipping behavior in urban transportation is shaped by a complex mix of social, economic, and temporal factors. This project explores tipping patterns in New York City's taxi and for-hire vehicle services using 2023 trip data provided by the Taxi & Limousine Commission (TLC), supplemented with neighborhood-level median income data.

Our goal is to understand whether and how tipping is influenced by the socioeconomic context of pickup and dropoff neighborhoods. We examine trip-level attributes and geographic income disparities across NYC to predict tip amounts and uncover patterns of urban mobility compensation.

- **Dependent variable**: Tip amount or tip percentage.\
- **Independent variables**: Trip distance, fare, tip percentage, hour, day of week, season, pickup and dropoff locations, neighborhood median income, taxi type.

## Data Retrieval

### Primary Dataset: NYC Taxi & Limousine Commission Trip Record Data

- Covers yellow taxi, green taxi, for hire vehicles trips across New York City during 2023.\
- Publicly available through the NYC Open Data platform.\
- Each row represents a single trip with details including pickup/dropoff times and locations, fare amount, tip amount, and payment type.

### Secondary Dataset: American Community Survey (ACS)

- Table: B19001 -- Household Income in the Past 12 Months (2023)\
- Retrieved from the U.S. Census Bureau's ACS 1-Year Estimates\
- Used to extract median income at the neighborhood (PUMA) level.\
- Income data was spatially joined to pickup/dropoff zones based on TLC taxi zones.

## Data Overview

After cleaning and preprocessing, we used trip data from four months in 2023 (as specified in our code). Each processed dataset contains:

- Approximately 21 columns, including engineered features such as tip percentage and time-of-day variables.\
- The datasets vary in size depending on the month but each contains hundreds of thousands of rows.\
- Trips with zero fare, zero tip, or missing location data were excluded.\
- Geographic income data was merged using spatial joins to match taxi zones to median income estimates.

## Libraries

The following Python libraries are required to run the analysis and modeling process:

``` python
import numpy as np
import pandas as pd
import statsmodels.api as sm
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor
import geopandas as gpd
```

------------------------------------------------------------------------

## Model Specification

To predict tipping behavior in NYC taxi rides, we implemented supervised machine learning models---specifically, Linear Regression and Random Forest Regression. Linear Regression provides a straightforward and interpretable baseline for understanding the relationship between tipping and variables like fare, distance, and neighborhood income. It helps identify whether a linear relationship exists between predictors and the tip amount or percentage.

Random Forest is a more flexible, non-linear model that can capture complex interactions between features. It is particularly well-suited for our dataset, which includes both numerical and categorical variables, and helps address potential overfitting through ensemble learning. Using both models allows us to compare performance and better understand the underlying patterns in tipping behavior.

We evaluate model performance using Root Mean Squared Error (RMSE) and R² Score to assess accuracy and explanatory power. These metrics help determine how well the models predict tipping based on socioeconomic and trip-related features.

### Feature Overview

| Feature | Description | Type |
|------------------|-------------------------------------|------------------|
| Distance | Total distance of the trip (in miles) | Numerical |
| Fare | Total fare amount for the trip | Numerical |
| Tip Percentage | Tip as a percentage of the fare | Numerical |
| Hour | Hour of the day when the trip started | Categorical |
| Day of Week | Day of the week (e.g., Monday, Tuesday) | Categorical |
| Pickup Location | TLC taxi zone where the trip started | Categorical |
| Dropoff Location | TLC taxi zone where the trip ended | Categorical |
| Neighborhood Income | Median household income of the pickup/dropoff neighborhood | Numerical |
| Taxi Type | Type of taxi (Yellow, Green, Uber, Lyft) | Categorical |

------------------------------------------------------------------------

## Methodology & Tools

To associate socioeconomic context with taxi trips, income data was merged with trip records by matching location identifiers to neighborhood-level income estimates. After cleaning and filtering the data to remove outliers and missing values, models were trained on a subset of trip data. Exploratory Data Analysis (EDA) was conducted using visualization libraries such as Matplotlib and Seaborn, allowing the team to identify trends, detect anomalies, and better understand the relationships between variables.

------------------------------------------------------------------------

## Citations

- NYC Taxi & Limousine Commission: <https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page>\
- U.S. Census Bureau, American Community Survey: <https://data.census.gov/table/ACSDT1Y2023.B19001?q=B19001+&g=040XX00US36$7950000,36$9700000>
1,897 changes: 1,897 additions & 0 deletions taxi_model_f.ipynb

Large diffs are not rendered by default.