You probably know already, but you absolutely cannot skip doses of an SSRI, which includes fluvoxamine, unless you've cleared things with your doctor. Even if you're taking a low dose the withdrawal is really unpleasant and can be physically dangerous. Luvox has a short half life and the withdrawal hits harder than some others too. Apparently this isn't always the case but I would not test things out just to see.
Here's an article on medical approaches to the sexual dysfunction caused by SSRIs, and it does suggest switching to or adding Wellbutrin (and Viagra), as well as lower doses and a "holiday" from the drugs (I still really, really wouldn't recommend this). Probably the best suggestion there is timing your dose to be after sex, or toying in your case, so your levels of the drug are lowest when you want to toy. It's at least worth trying.
Naltrexone can cause libido issues too, but can also induce erections and help with orgasms, at least anecdotally, so it's hard to tell. Spiro definitely does, it's an anti-androgen (reduces testosterone, in simple terms) to the point that trans women take it as a hormone blocker. Is it possible that you could move to a topical solution if not another medication? A cream would have fewer effects on your whole system, I think.
I dunno, personally SSRIs (200mg Zoloft) haven't tanked my interest in sex (physical or mental), just my ability to get off easily. I've mostly brute forced it with more powerful toys, TBQH, which might not be ideal. If you don't use a strong vibrator though, I do think it's worth trying. At least for me something high powered and rumbly/with deep vibrations works better than alternatives, and I've read other people on meds saying the same thing.