Your report is rather unclear: you say it "won't let you adventure" in the tower ruins, but you don't explain what "won't let" means. My initial assumption was that it's similar to your many previous rude bug reports about KoLmafia not letting you go to the Hole in the Sky if it fails to detect quest items, for example. Fernswarthy's key is a quest item, so, it seemed likely that this was more of the same.
Except, KoLmafia knows nothing special about Fernswarthy's key and the tower ruins. There is no code which restricts you from going to the ruins based on having or not having a key; if you select Fernswarthy's Ruins as an adventuring location, KoLmafia dutifully submits requests to go to snarfblat=22. There is nothing in the code which will prevent you from trying to go to the ruins, whether or not you have the key, whether or not you have visited the guild, whether or not you are even high enough level to have been given the quest. Therefore, I have absolutely no idea what you mean when you say KoLmafia "won't let you adventure" there.
I suppose this could be another case like adventuring Above the Beanstalk: I discovered, years ago, that if you didn't actually visit beanstalk.php - the container document that shows the Fantasy Airship and such - that you couldn't successfully adventure in the Airship. Therefore, we enable that zone by visiting the container document the first time you attempt to visit the Fantasy Airship. Perhaps Fernswarthy's Ruins require similar arming.
Given your history of verbal abuse and complaints at KoLmafia's special handling of adventuring zones, I'm not inclined to spend any time whatsoever to add more such special code to deal with an adventure zone which is currently blissfully untainted by such.
