Git Product home page Git Product logo

sameorigin's Introduction

@cawfree/sameorigin

๐Ÿค– ๐Ÿงช Masquerade as if you were their own frontend.

sameorigin generalizes the process of Same-Origin-Resource-Crossing, which allows you to work around the domain-based restrictions centralized services use to protect their APIs from third parties.

๐Ÿš€ Getting Started

You can install sameorigin via yarn:

yarn add @cawfree/sameorigin

Next, declare the domain you wish to squat on. In the following example, let's assume I want to bypass the CloudFlare restrictions on the Blur Marketplace:

import {sameorigin} from '@cawfree/sameorigin';

const [axios, {close}] = await sameorigin({
  // Define the website url that has access permissions and generates some requests.
  squatURL: 'https://blur.io/airdrop',
});

const {data} = await axios({
  // Important! You must declare the baseURL of the API you intend
  // to target. This is because it is used to isolate requests you
  // intend to hijack for your own purposes.
  baseURL: 'https://core-api.prod.blur.io/v1',
    
  // GET https://core-api.prod.blur.io/v1/prices
  url: '/prices',
  method: 'get',
});

// Once finished, you'll need to close your client to prevent
// memory leaks.
await close();

Warning

Some interfaces defend against tools like sameorigin by making an analysis of the runtime window. If you find your requests are being rejected, please instantiate sameorigin using headless: false as a workaround.

๐Ÿค” How does it work?

When making a call to sameorigin, we allocate an instance of puppeteer in the background which is used to capture requests and serve as a trusted origin for API requests to originate from.

We make the page available via an express server which manages the life cycle of pages, captures requests which satisfy CloudFlare's protections and dynamically inject them with custom query data.

This process is masked behind the returned axios client. The underlying process of squatting on pages, hijacking requests and returning the data as if it were a conventional fetch request are abstracted away from the caller.

โœŒ๏ธ License

CC0-1.0

sameorigin's People

Contributors

cawfree avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

wxf-wangxufeng

sameorigin's Issues

Adding a proxy option

Is there a chance to add to the 'sameorigin' function a proxy optoin with ip/port and some auth method ?

Errors with OpenSea call

Hello,
Trying to use this for OpenSea Open Graph but I have errors.

const [axios, { close }] = await sameorigin({
  squatURL: 'https://opensea.io/fr/collection/mutant-ape-yacht-club/activity',
});

const { data } = await axios({
  baseURL: 'https://opensea.io/__api',
  url: '/graphql',
  method: 'post',
  headers: {
    'Content-Type': 'application/json'
  },
  data: '{"id":"CollectionInfoPollingQuery","query":"query CollectionInfoPollingQuery(\n  $collection: CollectionSlug!\n) {\n  collection(collection: $collection) {\n    ...CollectionInfoInnerPoll_data\n    id\n  }\n}\n\nfragment CollectionInfoInnerPoll_data on CollectionType {\n  ...collection_poll_stats\n}\n\nfragment collection_poll_stats on CollectionType {\n  statsV2 {\n    totalListed\n  }\n  collectionOffers(first: 1) {\n    edges {\n      node {\n        perUnitPriceType {\n          unit\n          symbol\n        }\n        id\n      }\n    }\n  }\n}\n","variables":{"collection":"mutant-ape-yacht-club"}}'
});

The request return an error :

data: '<!DOCTYPE html>\n' +
      '<html lang="en">\n' +
      '<head>\n' +
      '<meta charset="utf-8">\n' +
      '<title>Error</title>\n' +
      '</head>\n' +
      '<body>\n' +
      '<pre>SyntaxError: Unexpected token &quot; in JSON at position 0<br> &nbsp; &nbsp;at JSON.parse (&lt;anonymous&gt;)<br> &nbsp; &nbsp;at createStrictSyntaxError (C:\\Users\\Romain\\Sites\\test\\node_modules\\body-parser\\lib\\types\\json.js:169:10)<br> &nbsp; &nbsp;at parse (C:\\Users\\Romain\\Sites\\test\\node_modules\\body-parser\\lib\\types\\json.js:86:15)<br> &nbsp; &nbsp;at C:\\Users\\Romain\\Sites\\test\\node_modules\\body-parser\\lib\\read.js:128:18<br> &nbsp; &nbsp;at AsyncResource.runInAsyncScope (node:async_hooks:203:9)<br> &nbsp; &nbsp;at invokeCallback (C:\\Users\\Romain\\Sites\\test\\node_modules\\raw-body\\index.js:238:16)<br> &nbsp; &nbsp;at done (C:\\Users\\Romain\\Sites\\test\\node_modules\\raw-body\\index.js:227:7)<br> &nbsp; &nbsp;at IncomingMessage.onEnd (C:\\Users\\Romain\\Sites\\test\\node_modules\\raw-body\\index.js:287:7)<br> &nbsp; &nbsp;at IncomingMessage.emit (node:events:513:28)<br> &nbsp; &nbsp;at endReadableNT (node:internal/streams/readable:1359:12)</pre>\n' +
      '</body>\n' +
      '</html>\n'
  }

Do you have any ideas to fix thix ?

possibility of the the web app that you're squatting blocking you?

First of all many thanks for this. It's opened up new alpha for me. However I was wondering if I send an abnormal amount of a certain API call, is there any way that the hosts of the web app catch on to this and alter the webapp to stop this workaround?

It would be very advantageous for me to be calling an endpoint maybe 20 or 30 times a second. But if that increases the probability of the alpha disappearing then I would rather content myself with something slower that would more resemble a normal user. Thoughts or ideas? Thanks!

error: unable to find matching request for baseUrl

hey there! i'm trying out the steps as per README. coming across an error (inserting response data html)

Error: Unable to find matching request for baseUrl "https://core-api.prod.blur.io".
    at /Users/aster27/Desktop/trade-bot/node_modules/@cawfree/sameorigin/dist/server/createServer.js:68:23
    at Layer.handle [as handle_request] (/Users/aster27/Desktop/trade-bot/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/Users/aster27/Desktop/trade-bot/node_modules/express/lib/router/index.js:328:13)
    at /Users/aster27/Desktop/trade-bot/node_modules/express/lib/router/index.js:286:9
    at param (/Users/aster27/Desktop/trade-bot/node_modules/express/lib/router/index.js:365:14)
    at param (/Users/aster27/Desktop/trade-bot/node_modules/express/lib/router/index.js:376:14)
    at Function.process_params (/Users/aster27/Desktop/trade-bot/node_modules/express/lib/router/index.js:421:3)
    at next (/Users/aster27/Desktop/trade-bot/node_modules/express/lib/router/index.js:280:10)
    at cors (/Users/aster27/Desktop/trade-bot/node_modules/cors/lib/index.js:188:7)
    at /Users/aster27/Desktop/trade-bot/node_modules/cors/lib/index.js:224:17

thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.