Handling American / UK spelling plus plurals in Sphinx - sphinx

Question

We need to have all these terms match each other and are running into difficulty
orthopaedic, orthopedic, orthopaedics, orthopedics
At the moment we are dealing with most other plurals using morphology stem_en
This is our current wordforms entry for this group (the pair is duplicated in reverse or
else it only works one way)
orthopaedic > orthopedic
orthopedic > orthopaedic
orthopedics > orthopaedics
orthopaedics > orthopedics
However "orthopedics" does not then match "orthopaedic" and we can't add another entry
"orthopaedic > orthopedics" because "orthopaedic" is already present and will throw an
error when indexing.
Any advice would be greatly appreciated

Answer 1

the pair is duplicated in reverse or else it only works one way
That's a bad idea! Putting it both way, will lead to issues (like you've found in fact!), you changing one to the other, so they wont match properly!
You need only one direction. Sphinx takes the left word, and actully stores the right in the index. So searching for the left and the right become interchangable. If you swap the words, then they nver get a chance to match.
The complication arrises because wordforms performs 'stemming exception' - ie a word in wordforms is NOT stemmed, so that means many word wont match. So you need to
perform stemming manually on the wordforms list, and
list all variations in your wordforms file, - to the same common word
Using your example above, would be something like
orthopaedic > orthopedic
orthopedic > orthopedic
orthopedics > orthopedic
orthopaedics > orthopedic
If the word did stem would have to do that, eg
bridge > bridg
bridges > bridg
bridging > bridg
etc
It vastly bloats your wordforms file, but it can be automated.