Member Avatar for Dani

Here at DaniWeb, we use a combination of a forum structure for top-level categories (web development, digital marketing, etc.), as well as a tags to denote specific technologies (php, etc.).

For many years, the moderation team wasn't actively doing a decent job moderating tags, and we now have a lot of older forum topics with incorrect tags, irrelevant tags, etc.

I would like to use AI where, given the list of available tags (it's a relatively finite list), it will properly tag all ~400,000 topics. As of right now, the only language I'm very comfortable with is PHP, because it's the only language I've coded in for the past 20 years.

What is the least expensive way to take on this task?

  • 2 Contributors
  • 5 Replies
  • 104 Views
  • 1 Week Discussion Span
  • Latest Post Latest Post by Dani

Recommended Answers

All 5 Replies

Member Avatar for Salem

There are a series of articles here on DW that use Python to interact with a variety of models.

If you can find them, maybe they'll give you an idea or two.

Member Avatar for Dani

So I've been tinkering yesterday and today and I have it recalculating tags for all 500,000 topics with model gpt-4.1 nano. I was previously using gpt-4o and gpt-4o mini which seemed to be a bit better quality, but I wanted to save some money. It looks like it's going to end up coming out around $500 in total. Once everything is recalculated, I'm going to compare it against what users have tagged with and do some SQL gymnastics to come up with something I'm happy with moving forward with.

Member Avatar for Dani

... and, just like that, I hit the maximum number of requests per day based on Tier 1. It's going to take a week to get to Tier 2 which doesn't have any max-per-date rate limiting. Oh well! Looks like this will just end up being a week long project.

Member Avatar for Dani

Well, it's been just over a week since I started this mini project, and all forum threads are now re-tagged. I've also even gone ahead and added some other little features such as tags now having sub-tags (marked with a colon). You can see how that looks here, for example: https://www.daniweb.com/tags/python

commented: Nice! +16
Member Avatar for Dani

Oh, and I should add, I was able to get all forum threads recategorized for just shy of $100 with OpenAI API.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.