forums.rs

Forums

API request limit

Quick find code: 351-352-79-65610194

Bowserkor
Sep Member 2007

Bowserkor

Posts: 148 Iron Posts by user Forum Profile RuneMetrics Profile
Good evening :)

Is there any way to increase the limit of requests we can have for the grand exchange API?

Currently we have 3048 Items in our database. We send a request every 5 seconds, so it keeps working properly without any problems. This makes it take 4 hours 14 minutes to once update the whole database.

We use the following link to receive the GE information:

http://services.runescape.com/l=1/m=itemdb_rs/api/catalogue/detail.json?item=<itemname>

Thanks in advance and sorry for the bad language in this case - I don't know enough about this topic myself to type it out properly ^_^ Admin of the german Schnupptrupp Fansite

21-Apr-2015 00:09:56

Lilroo503
Aug Member 2023

Lilroo503

Posts: 445 Silver Posts by user Forum Profile RuneMetrics Profile
I know that at least in the past, the suggested route to take was to NOT grab every item on a schedule. Instead, only grab information when a user requests that specific page and store that information for X amount of time.

This would mean that you are grabbing the data you need and saving it to your own database, then you are only making calls to your own database for the next 30 minutes, 1 hour, 3 hours, 24 hours... whatever time you feel is appropriate. It also means you aren't wasting both yours and Jagex' resources for items that your users do not care about. Once that time has expired and the page is called again, that's when you go out and make another call for the data.

It might not be what you're wanting to hear, but hopefully it makes sense and gives you another option to consider. Den Leader of the new fansite
Scapers' Den
(@ScapersDen).
Unique features include an Events Hub and JTalk, our JMod post viewer.

21-Apr-2015 19:00:14

Bowserkor
Sep Member 2007

Bowserkor

Posts: 148 Iron Posts by user Forum Profile RuneMetrics Profile
Thanks for your reply Lilroo.

The problem with this is, we are also making statistics regarding the prices of items. Maybe some items that are only listed on specific pages will not be updated for a longer timeframe then.

Isn't it the main purpose of the API the get the data you wish fast and easy from the respective source?

We currently request one price daily with the API, and it is kinda "cached" in our database to put the data in the statistics as well.

If we only request an items data when someone is opening a page with it, some items might get no update for a longer time, while others are frequently updated. This again will cause trouble regarding the statistics we are making.

I hope I got it right what I was told, still not the kind of stuff I'm working with unfortunatly :P Admin of the german Schnupptrupp Fansite

21-Apr-2015 22:51:00

Lilroo503
Aug Member 2023

Lilroo503

Posts: 445 Silver Posts by user Forum Profile RuneMetrics Profile
Ah, I see. You could always partially use the method I suggested above and only then have to grab the items which had not been touched that day. That would reduce your time, but by how much would depend on what your users are viewing. Still not ideal, I know.

While I'm doubtful Jagex will increase the API limit, I do hope that someday in the future they can update to something like an OAuth system where authenticated users can have higher limits and possibly access to additional features. I know I'm really dreaming now, but I'm still hoping that some day in the future we can have users choose to allow us access to more of their account's information... things like personalized D&D tracker, list of daily tasks, list of quests and completion status, list of friends and online status, etc. Nothing sensitive, no account changes; just info where we could customize a user's experience more when visiting a fansite and allow them to directly see some of their RS info without being in game. This could encourage users to log in if they hadn't planned on it but see something they like.

Sorry, I may have rambled on a bit. Best of luck with your problem! Den Leader of the new fansite
Scapers' Den
(@ScapersDen).
Unique features include an Events Hub and JTalk, our JMod post viewer.

22-Apr-2015 22:55:55

Das Wanderer
Jan Member 2011

Das Wanderer

Posts: 5,208 Rune Posts by user Forum Profile RuneMetrics Profile
Not sure if this is in anyway helpful but semi relevant.

When I went to the last RuneFest I had a chat with Mod Boko about some updates to the APi's and said make a thread and at somepoint they might get round to it, see it as a reference point if he ever has spare time :p

Thread

I think Lilroo has the right approach, cache pages users access and use a cron job to update the rest once a day? Lead Developer on the fansite RuneHints (@runehints)
RF 2011 & 2014 & 2015 Attendee

23-Apr-2015 18:09:57

Matonym

Matonym

Posts: 572 Steel Posts by user Forum Profile RuneMetrics Profile
Increase in GE API requests limits would help us a lot as well. The limit is something we've been struggling with for quite some time now.
Head of Czecho-Slovak Community

Czecho-Slovak Forces and RskoCz - language fansites
Find me on Twitter , Reddit and Twitch as
@CSF_RS

01-May-2015 14:56:43

ZPUN
Nov Member 2022

ZPUN

Posts: 84 Iron Posts by user Forum Profile RuneMetrics Profile
The current API is not designed for the function you're looking to create, and really shouldn't be used for such a thing. Reducing the limit could work but polling every single item individually would cause a lot of stress on whatever server they're using. It's inefficient.

Jagex needs to instead update their API to return matrices of data with a single request, via a search query or an index.
/r/runescape
mod - Official Jagex Approved Fansite

05-May-2015 19:02:51

Oneiromancer
Aug Member 2009

Oneiromancer

Posts: 4,941 Adamant Posts by user Forum Profile RuneMetrics Profile
I believe a script on RuneScape Wiki uses 2-3 seconds between requests for the API, I'd have to check to be sure.

As far as improvements to the API goes, it would be great to be able to query data in chunks like 10, 20, 50, etc. items at a time. Something like /exchange.ws?item=id&limit=50&props=name|price|id|examine

24-May-2015 12:55:54

Quick find code: 351-352-79-65610194 Back to Top