LaVOZs

The World’s Largest Online Community for Developers

'; python - For loop on API - LavOzs.Com

I have a list of around 28K numbers in a list named "y" and I am running a for loop on API to send Messages but this takes a lot of time (to be exact 1.2797 seconds per call)

Code:

import timeit

start = timeit.default_timer()

for i in y:
    data = {'From': 'XXXX', 'To': str(i),
            'Body': "ABC ABC" }
    requests.post('https://xxxx:xx@api.xxx.com/v1/Accounts/xxx/Sms/send',data=data)

stop = timeit.default_timer()
print('Time: ', stop - start)   

How can I reduce the time for this ?

Asyncio or Multithreading are the two possible solutions to optimize your code, and both basically do the same under the hood:

Threaded

import timeit
import threading
import time

y = list(range(50))


def post_data(server, data, sleep_time=1.5):
    time.sleep(sleep_time)
    # request.post(server, data=data)


start = timeit.default_timer()

server = 'https://xxxx:xx@api.xxx.com/v1/Accounts/xxx/Sms/send'

threads = []
for i in y:
    # if you don't need to wait for your threads don't hold them in memory after they are done and instead do
    # threading.Thread(target, args).start()
    # instead. Especially important if you want to send a large number of messages
    threads.append(threading.Thread(target=post_data,
                            args=(server, {'From': 'XXXX', 'To': str(i), 'Body': "ABC ABC"}))
    threads[-1].start()

for thread in threads:
    # optional if you want to wait for completion of the concurrent posts
    thread.join()

stop = timeit.default_timer()
print('Time: ', stop - start)

Asyncio

Referring to this answer.

import timeit
import asyncio
from concurrent.futures import ThreadPoolExecutor

y =  list(range(50)
_executor = ThreadPoolExecutor(len(y))

loop = asyncio.get_event_loop()

def post_data(server, data, sleep_time=1.5):
    time.sleep(sleep_time)
    # request.post(server, data=data)

async def post_data_async(server, data):
    return await loop.run_in_executor(_executor, lambda: post_data(server, data))


async def run(y, server):
    return await asyncio.gather(*[post_data_async(server, {'From': 'XXXX', 'To': str(i), 'Body': "ABC ABC"})
                                  for i in y])


start = timeit.default_timer()

server = 'https://xxxx:xx@api.xxx.com/v1/Accounts/xxx/Sms/send'

loop.run_until_complete(run(y, server))

stop = timeit.default_timer()
print('Time: ', stop - start)

When using an API that does not support asyncio but would profit from concurrency, like your use-case, I'd tend towards using threading as it's easier to read IMHO. If your API/Library does support asyncio, go for it! It's great!

On my machine with a list of 50 elements the asyncio solutions clocks in at 1.515 seconds of runtime while the threaded solution needs about 1.509 seconds, when executing 50 instances of time.sleep(1.5).

Related
Accessing the index in 'for' loops?
How do I loop through or enumerate a JavaScript object?
Emulate a do-while loop in Python?
A 'for' loop to iterate over an enum in Java
Loop through an array in JavaScript
Get loop count inside a Python FOR loop
Iterating over dictionaries using 'for' loops
Why does python use 'else' after for and while loops?
How to loop over files in directory and change path and add suffix to filename
Issues with thread and dynamic list in python