Warning: Paging is ignored

[jason-nyc]

DUO introduced paging after this code was written. So for something like admin/v1/users, it's only going to get the first 100 users because the next_offset is ignored in the response (indicating more users exist).

[Ramtecspdx]

Interesting, I think I have read something about this. I am going to test tomorrow and let you know what I find.

[Ramtecspdx]

Jason I forgot to mention, did you get the same results with postman? I have been testing along with postman making sure I see identical results. It could be that some parameter is not being passed.

[jason-nyc]

i don't know much about postman. i used this code to get the paging stuff to work, but i'm only using it to get a list of all users, so not sure if it works in all cases:

` $DuoRequest = Convertto-DUORequest -DuoMethodPath $path -Method $method -ApiParams $ApiParams
$Response = Invoke-RestMethod @DuoRequest
If ($Response.stat -ne 'OK') {
Write-Warning 'DUO REST Call Failed'
Write-Warning "APiParams:"+($APiParams | Out-String)
Write-Warning "Method:$method Path:$path"
}
$Output = $Response | Select-Object -ExpandProperty Response
### See if there are more pages
While ($Response.metadata.next_offset)
{
$APiParams = @{'limit'=100; 'offset'=$Response.metadata.next_offset}
$DuoRequest = Convertto-DUORequest -DuoMethodPath $path -Method $method -ApiParams $APiParams
$Response = Invoke-RestMethod @DuoRequest
$Output += $Response | Select-Object -ExpandProperty Response
}

Write-Output $Output`

[Ramtecspdx]

I see, it was hectic today... I will do the testing I promised tomorrow. The reason I haven't come across this issue is because I have been working in a dev environment with a handful of users.

[Ramtecspdx]

What I learned I that you can set the maximum limit to 300. But the pagination is going to be Max number of licenses / 300 that will give you the iterations needed if you were to put it in a loop and sent it to a csv file with -append. But there must be another way. Still reading and this time testing.

[jason-nyc]

I don't think you should be computing the number of iterations and really you shouldn't adjust the page size to 300. Just use the default page size and then keep looping for more pages until there aren't any (as returned by next_offset).

That's what my code is doing in the while loop.

See if there are more pages

While ($Response.metadata.next_offset)
{

append more results using +=

}

[Ramtecspdx]

Yeah, I will look at it again. Thanks for keeping me on track.:)

…

On Tue, Jan 18, 2022 at 12:32 PM jason-nyc ***@***.***> wrote: I don't think you should be computing the number of iterations and really you shouldn't adjust the page size to 300. Just use the default page size and then keep looping for more pages until there aren't any (as returned by next_offset). That's what my code is doing in the while loop. See if there are more pages While ($Response.metadata.next_offset) { append more results using += } — Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEC7W2JRZVMSXP5SZFTM77DUWXE45ANCNFSM5LRAUAGA> . You are receiving this because you commented.Message ID: ***@***.***>

[Ramtecspdx]

Yeah, I see what you did with the while loop and the append response.That is very clever. I will test tomorrow.

[Ramtecspdx]

I don't think you should be computing the number of iterations and really you shouldn't adjust the page size to 300. Just use the default page size and then keep looping for more pages until there aren't any (as returned by next_offset).

That's what my code is doing in the while loop.

See if there are more pages

While ($Response.metadata.next_offset) {

append more results using +=

}

it works pretty good, I tested setting the pagination to 300 and the execution time in miliseconds was faster by 28k miliseconds for the amount of records pulled.. I tested it getting the duo phones and loop for pagination also works. I suspect that there might be another way using the parameter total_objects ... this is what I get when I run it from Postman. {"metadata": {"next_offset": 100, "total_objects": XXXXX}, "response": [{"alias1": "yyyyy" ...
Good job Jason!