This would be added to the question approval process. When a new question is added, it will be compared to all the other questions. If it is 80% or more similar, it will be be listed as such to the moderator approving the question. This would allow the moderator to compare and prevent duplicates.
(preferable option)
Alternatively, create a command that finds all duplicates (again, 80% or more similar in characters), then create a new text file that lists them. The administrator would then need to go through, and pick which questions to delete.
Things to consider: How much memory will be used to scan the questions? How long will it take?
This would be added to the question approval process. When a new question is added, it will be compared to all the other questions. If it is 80% or more similar, it will be be listed as such to the moderator approving the question. This would allow the moderator to compare and prevent duplicates.
(preferable option)
Alternatively, create a command that finds all duplicates (again, 80% or more similar in characters), then create a new text file that lists them. The administrator would then need to go through, and pick which questions to delete.
Things to consider: How much memory will be used to scan the questions? How long will it take?