Git Product home page Git Product logo

Comments (5)

FelipeSBarros avatar FelipeSBarros commented on June 17, 2024 1

@cuducos se puder, da uma olhada nesse erro que tenho encontrado...

from crossfire.

FelipeSBarros avatar FelipeSBarros commented on June 17, 2024

Usando o Client sincrônico, pude identificar que o mesmo erro acontece.
E é gerado a partir da página 61:

from crossfire.clients import Client
client = Client()
client.occurrences('813ca36b-91e3-4a18-b408-60b27a1942ef')
Loading pages: 0page [00:00, ?page/s]
Loading pages:   1%|          | 1/186 [00:01<04:52,  1.58s/page]
Loading pages:   1%|          | 2/186 [00:03<05:50,  1.90s/page]
Loading pages:   2%|▏         | 4/186 [00:03<02:15,  1.34page/s]
Loading pages:   3%|▎         | 5/186 [00:03<01:38,  1.83page/s]
Loading pages:   4%|▍         | 7/186 [00:04<00:58,  3.07page/s]
Loading pages:   5%|▌         | 10/186 [00:04<00:31,  5.55page/s]
Loading pages:   6%|▋         | 12/186 [00:04<00:43,  4.02page/s]
Loading pages:   8%|▊         | 14/186 [00:05<00:35,  4.89page/s]
Loading pages:   9%|▊         | 16/186 [00:05<00:27,  6.10page/s]
Loading pages:  11%|█         | 20/186 [00:05<00:17,  9.40page/s]
Loading pages:  12%|█▏        | 22/186 [00:06<00:28,  5.67page/s]
Loading pages:  14%|█▍        | 26/186 [00:06<00:19,  8.18page/s]
Loading pages:  15%|█▌        | 28/186 [00:06<00:16,  9.43page/s]
Loading pages:  16%|█▌        | 30/186 [00:06<00:16,  9.48page/s]
Loading pages:  17%|█▋        | 32/186 [00:07<00:21,  7.17page/s]
Loading pages:  18%|█▊        | 34/186 [00:07<00:19,  7.69page/s]
Loading pages:  20%|█▉        | 37/186 [00:07<00:18,  8.08page/s]
Loading pages:  21%|██        | 39/186 [00:08<00:22,  6.50page/s]
Loading pages:  22%|██▏       | 41/186 [00:08<00:20,  7.14page/s]
Loading pages:  23%|██▎       | 43/186 [00:08<00:20,  6.95page/s]
Loading pages:  24%|██▍       | 45/186 [00:08<00:18,  7.57page/s]
Loading pages:  25%|██▍       | 46/186 [00:09<00:18,  7.75page/s]
Loading pages:  25%|██▌       | 47/186 [00:09<00:17,  7.81page/s]
Loading pages:  26%|██▌       | 48/186 [00:09<00:17,  8.03page/s]
Loading pages:  26%|██▋       | 49/186 [00:09<00:17,  7.61page/s]
Loading pages:  27%|██▋       | 51/186 [00:09<00:13,  9.89page/s]
Loading pages:  28%|██▊       | 53/186 [00:09<00:12, 10.83page/s]
Loading pages:  30%|██▉       | 55/186 [00:09<00:12, 10.22page/s]
Loading pages:  31%|███       | 57/186 [00:10<00:12, 10.67page/s]
Loading pages:  32%|███▏      | 60/186 [00:10<00:08, 14.20page/s]
Loading pages:  33%|███▎      | 61/186 [00:24<00:08, 14.20page/s]Traceback (most recent call last):
  File "/opt/pycharm-community-2022.3.2/plugins/python-ce/helpers/pydev/pydevconsole.py", line 364, in runcode
    coro = func()
  File "<input>", line 1, in <module>
  File "/home/felipe/repos/crossfire/crossfire/clients/__init__.py", line 167, in occurrences
    occurrences = loop.run_until_complete(
  File "/home/felipe/.pyenv/versions/3.10.2/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/home/felipe/repos/crossfire/crossfire/clients/__init__.py", line 128, in occurrences
    return await occurrences()
  File "/home/felipe/repos/crossfire/crossfire/clients/occurrences.py", line 114, in __call__
    pages = await gather(*requests)
  File "/home/felipe/repos/crossfire/crossfire/clients/occurrences.py", line 95, in page
    return await self.page(url)
  File "/home/felipe/repos/crossfire/crossfire/clients/occurrences.py", line 85, in page
    occurrences, metadata = await self.client.get(url)
  File "/home/felipe/repos/crossfire/crossfire/clients/__init__.py", line 92, in get
    response.raise_for_status()
  File "/home/felipe/.cache/pypoetry/virtualenvs/crossfire-WqfJa-6U-py3.10/lib/python3.10/site-packages/httpx/_models.py", line 758, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '400 Bad Request' for url 'https://api-service.fogocruzado.org.br/api/v2/occurrences?idState=813ca36b-91e3-4a18-b408-60b27a1942ef&typeOccurrence=all&page=https%3A%2F%2Fapi-service.fogocruzado.org.br%2Fapi%2Fv2%2Foccurrences%3FidState%3D813ca36b-91e3-4a18-b408-60b27a1942ef%26typeOccurrence%3Dall%26page%3D16'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

Fazendo uma requisição para essa página em especial, o metadado, parece estar bem:

from crossfire.clients import Client

client = Client()
client.occurrences('813ca36b-91e3-4a18-b408-60b27a1942ef')
await client.get('https://api-service.fogocruzado.org.br/api/v2/occurrences?idState=813ca36b-91e3-4a18-b408-60b27a1942ef&typeOccurrence=all&page=61')

..., Metadata(page=61, take=50, item_count=9267, page_count=186, has_previous_page=True, has_next_page=True))

from crossfire.

FelipeSBarros avatar FelipeSBarros commented on June 17, 2024

Fazendo um teste mais:
Para confirmar não ser um problema do estado, testei para: 'b112ffbe-17b3-4ad0-8f2a-2038745d1d14'

from crossfire.clients import Client

client = Client()
client.occurrences('b112ffbe-17b3-4ad0-8f2a-2038745d1d14')

E o mesmo problema foi observado:

oading pages:   6%|▋         | 52/807 [00:13<02:25,  5.19page/s]
Loading pages:  32%|███▏      | 59/186 [18:24<39:36, 18.71s/page]
Loading pages:  33%|███▎      | 61/186 [09:28<19:24,  9.31s/page]
Loading pages:   7%|▋         | 57/807 [00:13<01:04, 11.57page/s]
Loading pages:   7%|▋         | 59/807 [00:29<01:04, 11.57page/s]Traceback (most recent call last):
  File "/opt/pycharm-community-2022.3.2/plugins/python-ce/helpers/pydev/pydevconsole.py", line 364, in runcode
    coro = func()
  File "<input>", line 1, in <module>
  File "/home/felipe/repos/crossfire/crossfire/clients/__init__.py", line 167, in occurrences
    occurrences = loop.run_until_complete(
  File "/home/felipe/.pyenv/versions/3.10.2/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/home/felipe/repos/crossfire/crossfire/clients/__init__.py", line 128, in occurrences
    return await occurrences()
  File "/home/felipe/repos/crossfire/crossfire/clients/occurrences.py", line 114, in __call__
    pages = await gather(*requests)
  File "/home/felipe/repos/crossfire/crossfire/clients/occurrences.py", line 95, in page
    return await self.page(url)
  File "/home/felipe/repos/crossfire/crossfire/clients/occurrences.py", line 85, in page
    occurrences, metadata = await self.client.get(url)
  File "/home/felipe/repos/crossfire/crossfire/clients/__init__.py", line 92, in get
    response.raise_for_status()
  File "/home/felipe/.cache/pypoetry/virtualenvs/crossfire-WqfJa-6U-py3.10/lib/python3.10/site-packages/httpx/_models.py", line 758, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '400 Bad Request' for url 'https://api-service.fogocruzado.org.br/api/v2/occurrences?idState=b112ffbe-17b3-4ad0-8f2a-2038745d1d14&typeOccurrence=all&page=https%3A%2F%2Fapi-service.fogocruzado.org.br%2Fapi%2Fv2%2Foccurrences%3FidState%3Db112ffbe-17b3-4ad0-8f2a-2038745d1d14%26typeOccurrence%3Dall%26page%3D62'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

Reparar que, o erro é gerado quando o request é feito para a pagina 59. E que para essa página e a página 61 indica que o total de páginas sería de 186, enquanto as requisições anteriores indicavam 807.

from crossfire.

cuducos avatar cuducos commented on June 17, 2024

Achei o bug.

O cliente de ocorrências tem um método page(self, number) que, como o nome do argumento diz, espera um número de página para fazer a requisição.

Acontece que quando a requisção falha, e vamos tentar novamente, estamos passando a URL para tentar novamente (ao invés do número da página para tentar novamente):

crossfire/crossfire/clients/occurrences.py

Line 80 in 60c5da5

return await self.page(url)

from crossfire.

FelipeSBarros avatar FelipeSBarros commented on June 17, 2024

Legal, @cuducos ! Valeu pela força!

from crossfire.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.