Some outdated records in mhq_terr_popunits wrt replacement ?

[florisvdh]

Using current master; d29cd9b.

> mhq_terr_datapath <- file.path(dirname(gitroot), "n2khab-sample-admin/data/mhq_terr/rapportage2025")

grts_ranking_draw == 46006

In mhq_terr_popunits we have:

> read_vc("mhq_terr_popunits", root = mhq_terr_datapath) %>%
  as_tibble() %>% filter(grts_ranking_draw == 46006)
# A tibble: 2 × 9
  point_code grts_ranking grts_ranking_draw sac   legacy_site type  polygon_id    phab source                    
  <chr>             <dbl>             <dbl> <lgl> <lgl>       <chr> <fct>        <int> <chr>                     
1 46006_1           46006             46006 TRUE  FALSE       4030  639290_v2020   100 assessment/habitatmap 2023
2 46006_2           50102             46006 TRUE  FALSE       4030  639351_v2020    90 assessment/habitatmap 2023

These first two lines are about the same drawn location, yet one line states it is valid 4030, the other has replaced it with a new location.

We can indeed see these contrasting states in the assessments dataset; one is from 2014, the other is from 2019:

> read_vc("mhq_terr_assessments", root = mhq_terr_datapath) %>%
  as_tibble() %>% filter(str_detect(point_code, "46006"))
# A tibble: 3 × 10
  assessment_date point_code type  is_present no_habitat assessment_source inaccessible not_measurable change_location db_ref
  <date>          <chr>      <fct> <lgl>      <lgl>      <chr>             <chr>        <chr>          <lgl>           <chr> 
1 2014-09-09      46006_1    4030  TRUE       FALSE      field assessment  NA           NA             FALSE           NA    
2 2019-08-22      46006_1    4030  FALSE      NA         field assessment  NA           NA             FALSE           NA    
3 2019-08-22      46006_2    4030  TRUE       FALSE      field assessment  NA           NA             TRUE            NA    

So clearly the conflict in mhq_terr_popunits comes from two different observation times.

@ToonHub am I right to assume that mhq_terr_popunits aims to represent only the currently valid units? If so, then probably 46006_1 x 4030 should be removed from it. In the other case, it will be needed to register the time interval for which each row is valid. Such approach will be needed in some form for inferences (knowledge about the sampling frame / population sizes in each cycle), but may not be the aim of mhq_terr_popunits.

grts_ranking_draw == 169137

We also have:

> read_vc("mhq_terr_popunits", root = mhq_terr_datapath) %>%
  as_tibble() %>% filter(grts_ranking_draw == 169137)
# A tibble: 3 × 9
  point_code grts_ranking grts_ranking_draw sac   legacy_site type    polygon_id    phab source                    
  <chr>             <dbl>             <dbl> <lgl> <lgl>       <chr>   <fct>        <int> <chr>                     
1 169137_1         169137            169137 TRUE  FALSE       2130_hd 669903_v2023    45 assessment/habitatmap 2023
2 169137_2        8557745            169137 TRUE  FALSE       2160    669903_v2023    30 assessment/habitatmap 2023
3 169137_3         533681            169137 TRUE  FALSE       2160    669906_v2023    97 assessment/habitatmap 2023

While the first row is about a match, not a replacement, the second and third rows are about two different replacements for the same type.

We can see that these replacements refer to 2022 (169137_2) and 2021 (169137_3):

> read_vc("mhq_terr_assessments", root = mhq_terr_datapath) %>%
  as_tibble() %>% filter(str_detect(point_code, "169137"))
# A tibble: 7 × 10
  assessment_date point_code type    is_present no_habitat assessment_source inaccessible not_measurable change_location db_ref
  <date>          <chr>      <fct>   <lgl>      <lgl>      <chr>             <chr>        <chr>          <lgl>           <chr> 
1 2021-08-24      169137_1   2120    FALSE      NA         field assessment  NA           NA             FALSE           169137
2 2021-08-24      169137_1   2160    FALSE      NA         field assessment  NA           NA             FALSE           169137
3 2021-08-24      169137_3   2160    TRUE       FALSE      field assessment  NA           NA             TRUE            169137
4 2022-10-11      169137_1   2120    FALSE      NA         field assessment  NA           NA             FALSE           169137
5 2022-10-11      169137_1   2160    FALSE      NA         field assessment  NA           NA             FALSE           169137
6 2022-10-11      169137_2   2160    TRUE       FALSE      field assessment  NA           NA             TRUE            169137
7 2024-07-03      169137_1   2130_hd TRUE       FALSE      field assessment  NA           NA             FALSE           NA  

Here I have no idea which replacement is to be regarded as the currently valid one, but should it not be the case that only one of 169137_2 and 169137_3 occurs in mhq_terr_popunits? Same considerations as above.

[ToonHub]

46006: indeed the most recent assessment indicates that 4030 is not present in 46006_1 anymore and therefore 46006_1 is not a population unit for 4030. Should be removed.

169137_1 was visited twice (by mistake) and two different replacements were performed.
I previously checked both replacements and they were not valid:

• 169137_3 is not within the same habmap polygon as 169137_1
• 169137_2 is within the same habmap polygon but the relative ranking within the polygon was most probably not respected

Both population units should be removed in mhq_terr_popunits.
They were not selected for monitoring cycle 2.

I will correct this.