@MistressEmelia for anyone curious - i checked out how Significant Terms works, reverse enginereed it a bit and decided it's not worth it to add ES to stack to such a simple website while i can do the same with pretty simple SQL query. On the other hand, this neo4j thing looks pretty exciting, i need to dig more about it! Might give us some interesting insights about the network
@MistressEmelia @cassolotl i've actually found something called "Significant terms" which seems to ideally fit our needs. Ideally i would love to move fully to ES from postgres to maximally simplify the stack but i'm not sure it's doable or even profitable. Need to read more about that :) Anyway - ES is a tool of choice for refactoring trending topics on Mastodon Tags Explorer, thank you for the ideas! 👍
@MistressEmelia @cassolotl thanks for the reply! The real question here is how to improve the algorithm to show the truly trending tags, not how to optimise the current algorithm. I'm thinking about basing it on: check the average taggings per hour and see if amount of taggings for last hour is at least, say, 20% higher than the average for last ~12 hours.
I need to test several solutions that comes to my mind - any suggestions are more than welcome tho !
Introducing Mastodon Tags Explorer - https://mastodon-tags-explorer.hcxp.co
You can read more about it here: https://discourse.joinmastodon.org/t/introducing-mastodon-tags-explorer-project/931