Skip to content

Added S3 inventory to storage purge#54

Open
mrtcode wants to merge 1 commit into
zotero:masterfrom
mrtcode:s3-purge
Open

Added S3 inventory to storage purge#54
mrtcode wants to merge 1 commit into
zotero:masterfrom
mrtcode:s3-purge

Conversation

@mrtcode

@mrtcode mrtcode commented May 25, 2017

Copy link
Copy Markdown
Member

It seems that the current purge scripts are quite old. There are some incompatibilities. For example getLocalFileInfo($info) function wants 'zip' paprameter, but we don't have it, because all file names are just hashes. So it means we need a modified getLocalFileInfo function that doesn't use 'zip' parameter in query?

There was a 'TODO' mentioning to use SSCAN when it will be available in phpredis. Now phpredis supports getting multiple results with spop. What was the initial behind SSCAN? performance?

Comment thread admin/storage_purge
$info->filename = $key;
//$info->zip = ?;

$info = Zotero_Storage::getLocalFileInfo($info);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a short window between getting file info and deleting it, where it can be updated by other user and result in a missing S3 file. One option would be to move this code in transaction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant