Hi guys very curious to know what your net profit % is if you are using llms?
Azlan Tariq
22 replies
Saw an article recently which suggested that people integrating gbt,gemini etc are facing this problem of managing the cost of these llms as they are very expensive
Replies
Chloe Adeline Foster@chloeadelinefoster
Profit margins heavily depend on use case and architecture. Running LLMs yourself cost effectively is still very challenging. For a low volume app, maybe 20-30% after infra costs. For high volume, likely single digit % or even losing money initially. Using APIs like OpenAI is more predictable, maybe 50%+ depending on pricing. Definitely a balancing act! What's worked for others?
Share
I’ve noticed the same thing. It seems like striking a balance between leveraging the power of LLMs and keeping costs in check is tricky
@jasper_james What i Feel that Model Compression will be a great way for the future but enterprises are having risk thoughts about open source llms lately
I’ve heard that optimizing LLM usage, like fine-tuning models for specific tasks, can help reduce costs
@kistine_sheffield Yeah using the old models for freemium or basic versions and the latest for the premium seems to be the go to strategy.
@kistine_sheffield @azlan_tariq Fine-tuning the model on the comprehensive dataset is what is going to be the future of development. We at Future4coding are pushing boundaries in AI in software development Do check us out
Our experience has been that integrating language models has helped us improve efficiency and accuracy, which indirectly boosts profits.
We outsource to an agency. They bring fresh ideas and handle all the platforms efficiently.
Since integrating LLMs like GPT and Gemini, we’ve seen our net profit increase by about 15%. However, managing the costs has been a challenge, so we need to continuously monitor and adjust our usage.
i'm experimenting with some techniques to lower the cost for my upcomig product
I read a research paper called Frugal Gpt there are some techniques to lower the cost.
do check it out
Absolutely, managing costs while leveraging the power of LLMs is a significant challenge. The balance involves several key factors:
1. Usage Efficiency: Optimizing how often and for what purposes LLMs are used can help control costs. For example, using LLMs for tasks that truly benefit from their capabilities rather than routine or low-value tasks.
2. Model Selection: Choosing the right size and type of model for specific applications can impact costs. Smaller models or fine-tuned versions may be more cost-effective for certain tasks compared to larger, more general models.
3. Infrastructure Costs: The cost of running LLMs, including cloud infrastructure and computational resources, can add up. Effective management of these resources, such as through batch processing or serverless options, can help keep expenses under control.
4. Scalability: Implementing solutions that can scale efficiently with demand without a linear increase in costs is crucial. Techniques like caching responses or using hybrid models can help in this regard.
5. Monitoring and Optimization: Regularly monitoring usage and performance metrics allows for fine-tuning and adjustments to reduce unnecessary expenditure.
Balancing these factors requires ongoing adjustments and strategic planning to maximize the benefits of LLMs while keeping costs in check.
@kennethmarko Yeah thats very insightful and you have made very good points there, using the old models for freemium or basic versions and the latest for the premium seems to be the go to strategy.
I’m exploring it for my business and wondering if it’s worth the investment.