How to treat unsexed composition data

[kellijohnson-NOAA]

Moving the thread from nwfscSurvey

Originally posted by @iantaylor-NOAA in pfmc-assessments/nwfscSurvey#66 (comment)
In terms of repeating for males and females and unsexed, I would love to get some better consensus on if/when it makes sense to

1. partition unsexed fish into male and female bins using the sex ratio of the observed fish in each length bin,
2. include unsexed fish as a separate vector of unsexed observations, or
3. ignore unsexed fish.

The current system where each STAT independently struggles to figure out what seems best in the absence of clear best practices is not awesome. If Option 1 is going to be available, we should think about where it makes the most sense to add that step and how to streamline it.

_Originally posted by @chantelwetzel-noaa in pfmc-assessments/nwfscSurvey#66 (comment)
I definitely would like to develop a method to partition the unsexed fish into either males or females based on sex ratio information. I went to add this a while ago and realized the logic needed for this was a bit more complicated and opted to set it aside for the time being. Of course, I did not put notes in the code as to why I thought it was going to be complicated and can't remember exactly what the stumbling blocks at the time were (could have just been a time issue).

However, I think we need to set this up in a manner that the default is to not apply a sex ratio. This at least forces people to really see the split of observations by sexed and unsexed if they have not been looking. I have encountered (and probably done) some really terrible applications where the sex ratio was applied by default resulting in really augmented length compositions versus to those that would have been created when not applying a sex ratio.

_Originally posted by @andi-stephens-NOAA in pfmc-assessments/nwfscSurvey#66 (comment)
The original PacFIN code allowed different types of partitioning the
unsexed, including:

no partitioning, just bin them as unsexed
calculate the bin-by-bin ratio and apply it
calculate the bin-by-bin ratio over a specified minimum size, small

fish are 50:50
use a user-specified single value or vector of values
allow a maximum size over which they are all assumed to be female.

Way too complicated!

[iantaylor-NOAA]

Here's my first attempt to map out a flowchart of decisions related to unsexed fish to spur discussion on this topic (editable slide here: https://docs.google.com/presentation/d/1RnfNzP6Nlyp_b4W2yZjbmBrvm5jUe0L8OJxO633dH1k/edit#slide=id.g12ab1bd588c_1_5):

[kellijohnson-NOAA]

An example of a "mucked up" figure where it is difficult to compare females, males, and unsexed fish from 2021 lingcod.

Problems

• Hard to determine if the top (i.e., female and unsexed) distributions are "stacked"
• Do not know what the text regarding sample sizes pertains to, are they of all rows of data or just sexed fish?
• Unclear from the figure when sexually dimorphic growth occurs (might want a vertical line)
• No information about all fish, where it might be of interest to have unsexed, male, female, all all on the same figure

[melissamonk-NOAA]

I don't think this takes up much more room than the current plot and we can add the underlying data as well, make them more transparent.

[kellijohnson-NOAA]

Would we need each of the distributions on its own line like MM's example or can they all use the same axis like below?

Is there information that we could be adding with colors here if we were to put them on different lines, basically this is an extension of what MM suggests but I am thinking that maybe we could use colors in the distribution to indicate something, like difference from the average?

[melissamonk-NOAA]

@kellijohnson-NOAA I would be afraid that putting them all on the same axis like the city mileage example would get crowded with the additional underlying data sources for each. Then again it might work. This is definitely the right direction, IMO, and should be "easy" to decide same axis or not with a single year of data.

[iantaylor-NOAA]

I'm happy to support whatever changes will make these figures easier to read.

A few issues that we deal with which aren't present in the examples which make our task more challenging:

• we have both observed and expected distributions in each case, so 6 total when there are unsexed fish in the mix
• we often have very spiky distributions
• we want to show the exact distribution, not a smoothed one
• we have almost identical distributions in some cases, at least on the ascending side where length-based selectivity is similar across sexes

I think separate lines as in the Irish poll example would be the best approach to clean things up when unsexed fish are present, but I'm not sure there would be one-size-fits-all solution to the spacing between lines relative the height of the distributions which allows some overlap but not too much.

[kellijohnson-NOAA]

@iantaylor-NOAA when you mention the spacing issue between lines are you wanting the line heights to match up across panels if there is or is NOT a group present? This is kind of the beauty of {ggplot2} in that you do not have to specify these things and they are just figured out for you unlike base graphics. You can make sure all factor levels are present though and not drop unused levels which makes their relative position the same.

[iantaylor-NOAA]

I'm OK dropping factors and having mismatched line positions among years.

I suppose my ideal would be line heights matching regardless of factors for cases like the lingcod example you posted where all three groups are present in almost all years. But in a case where there were three factors only in a rare year, it would be better not to waste space for the absent factor in all the other years.

In commenting about spacing between lines relative to the height of the distribution, I was thinking more about what seems to be controlled nicely by the scale parameter in {ggridges} as discussed here: https://cran.r-project.org/web/packages/ggridges/vignettes/introduction.html#:~:text=Finally%2C%20if%20you,work%20very%20well.)