« first day (4012 days earlier)      last day (637 days later) » 

1:37 AM
@anonymous2 Thanks! spacy seems to have done a good job. Chart is updated. It's interesting to see that there seem to be more starts for each given end than ends for each given start at every chunk length, but they approach parity as the chunk gets longer
 
 
2 hours later…
3:08 AM
Indeed that is interesting. I assume the ratio is top 10% for each category as well?
 
Yup. I might try different ratios later. I chose that one arbitrarily, but I think the pool of permutations being so different in scale at different lengths might make a single ratio unsuitable...
 
 
9 hours later…
12:28 PM
True
I may be taking it a step far but you could do a linear regression with % of words and chunk length as your independent variables
It would have the added advantage of correcting for the rounding in 10%
 

« first day (4012 days earlier)      last day (637 days later) »