Skip to content

Create GeocodeEvictions_DC2024.sas #88

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RAGarMir wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Create GeocodeEvictions_DC2024.sas #88

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
b70ad11
Create GeocodeEvictions_DC2024.sas
RAGarMir Oct 22, 2024
41f4019
Update GeocodeEvictions_DC2024
RAGarMir Oct 31, 2024
2b7c436
Update GeocodeEvictions_DC2024.sas
ptatian Nov 1, 2024
903e149
Add label for street_address
ptatian Nov 1, 2024
569790a
Update GeocodeEvictions_DC2024.sas
RAGarMir Nov 13, 2024
05c07b0
Update StreetAlt_GeoEvict_2024.txt
RAGarMir Nov 14, 2024
83b8f7f
Updated GeocodeEvictions_DC2024.sas & StreetAlt_GeoEvict_2024.txt
RAGarMir Nov 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 89 additions & 0 deletions Prog/2024/GeocodeEvictions_DC2024.sas
View file
Open in desktop

@RAGarMir RAGarMir Nov 13, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added if/then statement to correct "cancelled" Spelling

@ptatian ptatian Nov 14, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @RAGarMir. Unfortunately, this code does not work (that is, it does not make the change we want). Can you tell why?

@RAGarMir RAGarMir Nov 14, 2024 •
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ptatian I am not sure why I does not work. After running the code and looking at the outputs I assume that the if/then statement is in the wrong place and the line with the "Cancelled" disposition is filtered out at an earlier step.

Re: StreetAlt_GeoEvict_2024.txt
I updated the file to remove the line referencing LACKLAND

@ptatian ptatian Nov 14, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Condition statements in SAS are case-sensitive by default. If you write disposition = 'cancelled', that is only true if the value is all lower case. But, in the data the disposition values are uppercase (CANCELLED).

It's better, however, not to assume the case of the values. In a future data file they might be different. You can use the upcase() function to convert the values of disposition to all upper case for the comparison: upcase(disposition) = 'CANCELLED'.

And, then, you want to assign the new value in uppercase as well so that they are consistent. 'CANCELED'

@ptatian ptatian Nov 14, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the default StreetAlt file and found some other problem entries. I've fixed the file in the MAR library but for your StreetAlt_GeoEvict_2024.txt file, you will want to delete these rows as well.

MC KENNA,MCKENNA
SURCUM CORDA,SURSUM CORDA
SURSUM CURDA,SURSUM CORDA
VANDENBURG,VANDENBERG
YONT,YOUNT

Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
/**************************************************************************
Program: {GeocodeEvictions_DC2024}.sas
Library: {Requests}
Project: Urban-Greater DC
Author: Rodrigo Garcia
Created: 10/21/24
Version: SAS 9.4
Environment: Local Windows session (desktop)
GitHub issue: Issue #87

Description: Geocoding Eviction Data from OTA

Modifications:
**************************************************************************/

%include "\\sas1\DCdata\SAS\Inc\StdLocal.sas";

** Define libraries **;
%DCData_lib( Requests )
%DCData_lib( MAR )

**'Proc import' and 'proc print' data import csv file into SAS**;
proc import datafile="\\SAS1\dcdata\Libraries\Requests\Raw\2024\FY24DetailedData(records).csv"
out=evictions
dbms=csv
replace;

getnames=yes;
guessingrows=max;

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

guessingrows=max; makes Proc Import look at the entire file when deciding on variable types. Without this statement, the code was creating street_number as a numeric var, but it contains some character elements (e.g., "5337/5341") that were being read as missing values.

run;

proc print data=work.evictions (obs=10);
run;

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added (obs=10) to limit the printed output to 10 rows.


**Creates new data set with new street_address and zip_char variables **;
data evictions_addresses;
Set work.evictions (rename=(CASE__=Case_num));
street_address = catx( " ", street_number, street, type, quadrant );

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proc Import doesn't handle column names like "CASE #" ver well, so I used (rename=(CASE__=Case_num)) to rename the case number variable to a more natural name.



if not( missing( zip ) ) then zip_char = put( zip, z5.0 );
if UPCASE( disposition )='CANCELLED' then disposition= 'CANCELED';

informat _all_ ;
format _all_ ;
format EVICTION_DATE mmddyy10.;

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proc Import add lots of unneeded formats and informats. I use these first two statements to clear them. But then the date format is needed for EVICTION_DATE, so I restore that one.

label
street_address = "Full street address for eviction case"
Case_num = "Eviction case number"
;

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used a label statement to a descriptive label to street_address and Case_num. Ideally, we want to label all variables in permanent data sets. Do we have a code book or something that describes what all the fields are? If so, we should use that to add labels for the remaining variables.


run;

proc print data=work.evictions_addresses (obs=40);
id Case_num;
var street_address street_number street type quadrant zip_char;
run;

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I limited the rows printed with (obs=40) and also what colums are output using the ID and VAR statements, assuming that we were mainly interested in seeing the new street_address var and its components.



**Geocoding Macro**;
%DC_mar_geocode(
data = evictions_addresses,
staddr = street_address,
out = GeocodedEvictionsData,

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed ZIP codes from the geocoding. Since many addresses are missing ZIP codes, we were getting a lot of unmatched addresses. From the LOG, 333 unmatched.

    _________ Geocoding Summary _____________________________
    Address data:            WORK._DCG_INDAT
    Output data:             WORK._DCG_OUTDAT
    STREET lookup data:      MAR.GEOCODE_94_DC_M
    ZIP lookup data:         SASHELP.ZIPCODE
    Geocoding method:        Street level
    Run date:                31Oct2024
    Obs processed:           2,923
    Elapsed time:            00:00:03
    Obs per minute:          53,781
    Street matches:          2,555
    ZIP matches:             35
    Not matched:             333
    _________________________________________________________

It works better without the ZIP codes.


    _________ Geocoding Summary _____________________________
    Address data:            WORK._DCG_INDAT
    Output data:             WORK._DCG_OUTDAT
    STREET lookup data:      MAR.GEOCODE_94_DC_M
    CITY lookup data:        MAPSGFK.USCITY_ALL
    Geocoding method:        Street level
    Run date:                01Nov2024
    Obs processed:           2,923
    Elapsed time:            00:00:27
    Obs per minute:          6,478
    Street matches:          2,922
    City matches:            1
    Not matched:             0
    _________________________________________________________

This is a bit of a quirk in the geocoding macro.

streetalt_file = &_dcdata_l_path\Requests\Prog\2024\StreetAlt_GeoEvict_2024.txt
)

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use the macro variable &_dcdata_l_path to designate the local path name, C:\DCData\Libraries. For most people it is the same, but in the past we had machines with D: drives. &_dcdata_l_path resolves to the correct name, regardless of what it is.


**Macro used to finalize and save dataset**;
%Finalize_data_set(
data=GeocodedEvictionsData,
out=Evictions_2024_dc,
outlib=Requests,
label="Scheduled and executed evictions, 2024, DC",
sortby=Case_num eviction_date,

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I adjusted the output data set name and label. Since we will likely have more eviction data sets, the data set name should include the year.

/** Metadata parameters **/
printobs=0,
freqvars=year DISPOSITION ward2022,
revisions=%str(New file.)

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I supressed the printed listing of data (printobs=0,) and added frequency tables for three variables (freqvars=).

)

%Dup_check(
data=evictions_addresses,

@ptatian ptatian Nov 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this macro from our macro library to print out obs that have the same case numbers. As you can see, there are many and we need to be aware of that if we want to map or do other analysis with the data.

by=Case_num,
id=street_address eviction_date disposition,
listdups=Y
)


Loading