Clarify behavior of “after” parameter in reddit.Scanner.Listing

[J-Thompson12]

in reddit scanner.go you have the after var getting set to before. This is breaking pagination

func (s *scanner) Listing(path, after string) (Harvest, error) {
	return s.r.reap(
		path, map[string]string{
			"raw_json": "1",
			"limit":    "100",
			"before":    after,
		},
	)
}

[turnage]

This is intentional. Not necessarily defending the decision, but I believed at the time that Reddit was incorrect. “after” in the graw API means “chronologically after”. At this point it would be a bad idea to change since users and the graw package depend on these semantics. You can use ListingWithParams to set whatever parameters you need.

…

On Fri, Apr 3, 2020 at 8:10 AM Justin Thompson ***@***.***> wrote: in reddit scanner.go you have the after var getting set to before. This is breaking pagination func (s *scanner) Listing(path, after string) (Harvest, error) { return s.r.reap( path, map[string]string{ "raw_json": "1", "limit": "100", "before": after, }, ) } — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#71>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAWR64GBEPX4REYADF6IOC3RKX37RANCNFSM4L4ESEXQ> .

[turnage]

I’ll re-open this until the a comment about this is added to clear up confusion in the future. Unfortunately I think that’s the best that can be done without breaking users.

[J-Thompson12]

I just pulled in 200 mentions using pagination with after and it pulled in the second page with all comments createdUTC earlier than the first page

[turnage]

With which meaning of the parameter “after”? If you pass “after” to Reddit, you will get what is chronologically before. If you pass “after” to reddit.Scanner.Listing, I believe you will receive what comes chronologically after.

This is the way all of the streams in graw work.

[J-Thompson12]

i used the listingWithParams set to after and got the createdUTC times

end of page 1.             beginning of page 2
1585763973                1585762846
1585763157                 1585761938
1585763134                  1585761819
1585763072.                1585761806
1585763016.                 1585761502
1585763009.                 1585761462

[turnage]

If you use ListingWithParams, you are talking to directly to Reddit with your parameters, so that is expected.

…

On Fri, Apr 3, 2020 at 9:05 AM Justin Thompson ***@***.***> wrote: i used the listingWithParams set to after and got end of page 1. beginning of page 2 1585763973 1585762846 1585763157 1585761938 1585763134 1585761819 1585763072. 1585761806 1585763016. 1585761502 1585763009. 1585761462 — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#71 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAWR64B3XEFL77N6PT7ZBF3RKYCK3ANCNFSM4L4ESEXQ> .