Git Product home page Git Product logo

esp8266rawpackets's Introduction

#Demonstration of raw packet sending on the ESP8266 for the purpose of triangulation.

NOW 2.0.0 COMPATIBLE. see #7

Introduction

This project is geared for the end attempt at triangulation (Which it can't do currently) but, I have run into some useful things along the way. Right now, this allows us to connect to an AP at the same time as send/receive arbitrary packets as well as get timestamps on both the sending and receiving.

I use the tactic outlined here: https://github.com/ernacktob/esp8266_wifi_raw to override aaEnqueueRxq. I am compiling against 1.5.4. Any other version will require libpp.a to be patched. This mechanism allows us to intercet message packet receives by overriding that function. Once overridden, all broadcast and packets targeted for our host will roll on in! We can process them any way we want. see esp_rawsend.c. Keep in mind this isn't true promiscuous mode, as it does not receive packets that are unicasted to a specific node but not this node.

Sending raw packets is done by directly calling the esp wifi_send_pkt_freedom function. I had originaly written RawSendBuffer, but it seems less reliable. For this test, I send data packets to broadcast, but use a special tag to identify them as my packets "82668266" type.

By using the custom build script, custom.ld, either the NMI or the User interrupt can be overridden. The custom interrupt handler in this case looks for packets with the packet type 0x82668266. If so, it writes in the CCOUNT that happened when the interrupt fired. This isn't very good since if there are two packets that come in one after another, or the interrupt is in use for a different cause, it will make sure the packet is skipped, or has inaccurate time. If a packet is missing entirelly, the time code will be 0. It is important to always remember the timecode may be very inaccurate and can be thrown out.

As it stands, the ESPs expect to be connected to a wifi network and will send all 82668266 packets back to a host at a fixed ip via pUdpServer - this means the host that is expecting the packets should be on the connected wifi network at 192.168.1.113, port 9999. The packets returned contain the standard Espressif wifi stack header.

Collecting the data

We can now run a host at 192.168.1.113. This host can bind to UDP port 9999 and collect all data from all ESPs on the network. An example of this app can be found in toprecorder. Data can be redirected to a data file. Once recorded the data can be processed.

An enterprising young soul, should they choose could probably process the data in real time, if they so chose.

2D/3D Localization

This is where things get tricky. Unlike GPS where we know the time of sending a packet, we can only really trust the time of packet reception. Additionally, we don't know anything about the times on the sending nodes. The only thing we can do is know the time a node sent a packet on a receiver's local oscillator time - we can determine the send time on the initial constellation by subtracting out the distance from one node to another.

We can synchronize nodes's clocks to one another only when two nodes receive the same packet. We can then know they're at the same time (barring outliers due to the interrupt happening at the wrong time). Once we do this for a while, we can get a pretty good idea what the time on one node would mean for another node. Over a little longer while, we can start to find the clock skew between the nodes and calibrate it back out.

If we wanted to find the location of a node within the constelaltion that we didn't know the locaiton of, we could transmit, then, look at the differential receive times on that node. This can tell us the relative distances of the transmitter to each node. There is no way to know the absolute distance unless we can be completely confident we can synchronize it's clock and that we can get transmit time.

There is a jitter of +/-4 clock ticks, so it would take many, many readings to get a good solid reading on what the true differential value would be.

It's important to note that in our example, all of the nodes are co-planar, so only 2D localization would be possible anyway. I recommend trying to resolve the location of node #5 to the constellation of 1-4 if you want to try to find something.

Why it's hard.

So, this is a hard question to answer. I think the biggest problem I have with my data is that I'm not very good at analyzing these sorts of things. Additionally, I think I'm fighting clock jitter. Though all crystals have natural speed differences which can be observed, easily in the +/- 5-10ppm range, the crystals also seem to have a random walk which makes it very, very hard to nail down times. Also, figuring out time synchronization from the differential systems is... hard :( - mathematically.

I've tried doing this a number of times, and I've checked my math by changing the "distance" between nodes to make sure the distances of the true constellation should match that of the data --- and often, they don't :(.

This data is outside and ideal. Multipath could mess things up! Def need too look at all the encoding schemes. Extra packets might be messing with the timing. Could consider sending data back via serial or Ethernet?

Also, maybe you could use rssi to help resolve?

esp8266rawpackets's People

Contributors

cnlohr avatar

Stargazers

Jason Thomas Horn avatar Felix Yustian Setiono avatar gvl610 avatar Elias Maier avatar  avatar  avatar Daniil avatar Juan Boirazian avatar Yea Chen avatar Darek Stojaczyk avatar Markus Stensiö avatar Agatha Uy avatar Hussein avatar  avatar  avatar mz289 avatar Stefan Andrei avatar  avatar Alexandre Esse avatar Nixon Nelson Paliakkara avatar Washiul Alam Sohan avatar min avatar asimos-bot avatar Soumen Das avatar Syg avatar Tim Speetjens avatar Vincent avatar /var/lib/libcrack.so.1 avatar  avatar  avatar Tim avatar  avatar Vakaris Žilius avatar  avatar  avatar  avatar Eugene Korneev avatar  avatar  avatar  avatar JD avatar Kevin.Y avatar Guy Dols avatar Gavin S avatar Colin Kuebler avatar Hsun Chao avatar  avatar Tianlan Zhou avatar Bowen Xia avatar latyas avatar  avatar libc0607 avatar Richard Mark Haes Ellis  avatar oddcod3 avatar Lorin Zahra-Newman avatar Pierre Gordon avatar samy kamkar avatar Åke Hedman avatar peiyong avatar rcortx avatar Alexander Kledal avatar  avatar  avatar GBP avatar Jack Wilsdon avatar  avatar Hassan Shifaz avatar H_P avatar Marcio Pamplona avatar Yifan Gu avatar  avatar Denis avatar TridentTD avatar Nikhil Jali avatar kaleng avatar Keshuai Xu avatar AI farmer avatar Ronan Gaillard avatar Badr Bouslikhin avatar nnamon avatar Cory Welch avatar Julius Gromyko avatar Marius Slavescu avatar Andres Aravena avatar Sean Stevens avatar  avatar Peter avatar Rodrigo Ratan avatar Evan  ™️ avatar Zhang, Chi avatar Allan Savolainen avatar Luke Alex Reeve avatar  avatar Stephan avatar  avatar Greg Price avatar  avatar Hasjim Williams avatar Arpan Halder avatar Harald Mueller avatar

Watchers

Brian Taylor avatar Christian Hochfilzer avatar Hasjim Williams avatar James Cloos avatar peiyong avatar Åke Hedman avatar Dan Drown avatar Ian Oliver avatar  avatar Dawei Shang avatar  avatar Stephan avatar David Fisher avatar Sam Ng avatar Arpan Halder avatar  avatar Hsun Chao avatar  avatar 仁二 avatar Narasing Sarwade avatar Shahruk Hossain avatar  avatar  avatar Vakaris Žilius avatar  avatar  avatar

esp8266rawpackets's Issues

image.elf section `.text' will not fit in region `iram1_0_seg' region `iram1_0_seg' overflowed by 224 bytes

HI,
I'm using ESP12-E when i tried to "make burn" cmd i'm getting following error:

/home/esp8266/vishal/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/../../../../xtensa-lx106-elf/bin/ld: image.elf section .text' will not fit in region iram1_0_seg'
/home/esp8266/vishal/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/../../../../xtensa-lx106-elf/bin/ld: region `iram1_0_seg' overflowed by 224 bytes
collect2: error: ld returned 1 exit status

so how i fix this issue?

Thanks

Dealing with Clock Wander

Copied from Youtube comment:
I took toprecorder/data10.txt data and looked specifically at offset and frequency differences between all the clocks:

Using .241's broadcasts as the "master" clock:

  • .179 is running 0.316ppm slower
  • .147 is running 2.218ppm slower
  • .169 is running 13.519ppm slower and jumped by 508.763 microseconds somewhere between .241's pktids 9254 and 9255
  • .213 is running 7.877ppm slower and jumped by 508.513 microseconds somewhere between .241's pktids 10715 and 10716

Removing the average frequency differences and the two clock jumps, I get this graph, which shows the clock wander: https://dan.drown.org/clocks/data10.png

Maybe trying to insulate the esp8266's from any airflow would lower their temperature changes, which should lower their clock wander.

Also, maybe using a PID control loop on each node would work to sync the frequencies. This is what I've done with NTP on the esp8266 along those lines: https://github.com/ddrown/Arduino_ClockPID

NTP uses round trip time to try to eliminate the phase offset due to one way latency. I'm not sure that would be needed for this application. Knowing the distances between the fixed points should make it possible to cancel out those terms in the equation.

Lastly, the rx and tx timestamp accuracy will add errors as well, but I haven't measured how accurate they are.

Packet injection and receiving raw wifi

Hi, this is not really an "issue", but I just saw your work on trying to get the ESP8266 to do some raw ieee82011 packet processing. As it turns out, I've been working on this for some time as well. It would be interesting for all those who have this same goal to keep in touch and show updates on their progress.

customnmi.s debugging

Hey. I forked this project in attempt to alter it into a RToF project here.
I had some success, but results still deviate ~100 CC's (20 000 packets gathered). I think the biggest delay happens on ESP "bouncer" (receive packet -> send it back routine). Could this routine be improved by writing a custom NMI? Or is sending a packet too long of a process for NMI? Maybe I could set a value with custom nmi and then send a packet on hardware timer if this value is set?

undefined reference to `wifi_set_raw_recv_cb'- In function myTimer

esp8266@esp8266-VirtualBox:~/vishal/esp82XX-basic$ make burn

/home/esp8266/vishal/esp-open-sdk/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc -mlongcalls -Os -I/home/esp8266/vishal/esp-open-sdk/sdk/include -Iesp82xx/include -I. -Iesp82xx/fwsrc -Iuser -DICACHE_FLASH -DWEB_PORT=80 -DCOM_PORT=7777 -DBACKEND_PORT=7878 -DSLOWTICK_MS=50 -DVERSSTR='"Version: v0.1-7-g06742-dev - Build 2017年 01月 03日 星期二 02:49:57 CST with -DICACHE_FLASH -DWEB_PORT=80 -DCOM_PORT=7777 -DBACKEND_PORT=7878 -DSLOWTICK_MS=50"' esp82xx/fwsrc/uart.c esp82xx/fwsrc/esp82xxutil.c esp82xx/fwsrc/flash_rewriter.c esp82xx/fwsrc/http.c esp82xx/fwsrc/commonservices.c esp82xx/fwsrc/http_custom.c esp82xx/fwsrc/mdns.c esp82xx/fwsrc/mfs.c user/custom_commands.c user/user_main.c -flto -Wl,--relax -Wl,--gc-sections -nostdlib -L/home/esp8266/vishal/esp-open-sdk/sdk/lib -L/home/esp8266/vishal/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/libgcc.a /home/esp8266/vishal/esp-open-sdk/sdk/lib/libmain.a /home/esp8266/vishal/esp-open-sdk/sdk/lib/liblwip.a /home/esp8266/vishal/esp-open-sdk/sdk/lib/libssl.a /home/esp8266/vishal/esp-open-sdk/sdk/lib/libupgrade.a /home/esp8266/vishal/esp-open-sdk/sdk/lib/libnet80211.a /home/esp8266/vishal/esp-open-sdk/sdk/lib/libwpa.a /home/esp8266/vishal/esp-open-sdk/sdk/lib/libphy.a /home/esp8266/vishal/esp-open-sdk/sdk/lib/liblwip.a /home/esp8266/vishal/esp-open-sdk/sdk/lib/libcrypto.a /home/esp8266/vishal/esp-open-sdk/sdk/lib/libpp.a /home/esp8266/vishal/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/libgcc.a -T /home/esp8266/vishal/esp-open-sdk/sdk/ld/eagle.app.v6.ld -B/home/esp8266/vishal/esp-open-sdk/sdk/lib -o image.elf
/tmp/cctpsX1W.ltrans1.ltrans.o:(.irom0.literal+0x34): undefined reference to wifi_set_raw_recv_cb' /tmp/cctpsX1W.ltrans1.ltrans.o: In function myTimer$2546':
:(.irom0.text+0x45): undefined reference to `wifi_set_raw_recv_cb'
collect2: error: ld returned 1 exit status
make: *** [image.elf] Error 1


wifi_set_raw_recv_cb( rx_func ); //function call

void attribute ((noinline)) rx_func( struct RxPacket * r, void ** v ) //defination

Using RawSendBuffer to send management frames?

Hey CnLohr
After discovering send_pkt_freedom can no longer send deauth packets / any other mgmt frames, I thought about using your RawSendBuffer (this should work, right?) but I cannot get it to work.
ppTxPkt and ppRegisterTxCallback are undeclared. Since I can see them in libnet80211/ieee80211.o, adding the declarations "extern ...." should be enough, right? I tried for quite a bit, but failed miserably, and cannot find them anywhere in your code either... Any idea?
Thanks :)

access points created but how to make further setup....i'm new to esp8266

Hi,
how should i make setup at router side. Actually executable toprecorder and process made but i don't know how to made setup at router side.

After "make burn" and "make netburn" it creates its own WiFi Access Point but i am new to esp8266 i don't know how to move forward to get packets.

Thanks.

printed ip not becoming 1 also due to printf() function's ESP12E getting reset.

untitled6
Hi @cnlohr ,

What should be the opmode? (i think so opmode=2 for access point for this program)

   **i have some doubts..**

in user_main.c at line 224--> if( !did_init && printed_ip ) printed ip not becomes 1 so it's not going inside loop. So when i'm using--> if( !did_init) i'm getting packages. see image i'm not using printed ip here.

also when i just added some printf(); function to check MAC addresses esp12e getting reset
rst cause:2, boot mode:(3,6).

Also at toprecorder i hardcoded ip address to 192.168.1.113 and when i run the program i'm not getting data..but my ESP's getting packets.

Thanks.

Can't get sdk with angus patch to work with pfalcon/esp-open-sdk

I know that this is several years old now, but I'm interested in experimenting with this. I followed this tutorial to get my setup working. Its basically pfalcons environment, but with a couple patches to help with a couple things. In order to get this recieve callback thing to work I need to use the sdk mentioned here. I tried make STANDALONE=n, as described in the readme for pfalcons environment, and to modify the make file:

CC=xtensa-lx106-elf-gcc -I$(DIR)/include -L$(DIR)/lib

LDLIBS=-nostdlib -Wl,-Map=output.map -Wl,--start-group -lc -lhal -lpp -llwip -lphy -lnet80211 -lwpa -lmain -Wl,--end-group -lgcc

CFLAGS= -I. -mlongcalls

LDFLAGS=-T$(DIR)/ld/eagle.app.v6.ld

where DIR is the path to the sdk. This worked when I set DIR to the file "ESP8266_NONOS_SDK-2.1.0-18-g61248df", that you get when you make in pfalcons environment, but when I try to set DIR to the sdk with angus patch, it returns with this

xtensa-lx106-elf-gcc -I/media/lysander/Data/Workspace/ESP8266/ESP8266_NONOS_SDK/include -L/media/lysander/Data/Workspace/ESP8266/ESP8266_NONOS_SDK/lib -T/media/lysander/Data/Workspace/ESP8266/ESP8266_NONOS_SDK/ld/eagle.app.v6.ld main.o -nostdlib -Wl,-Map=output.map -Wl,--start-group -lc -lhal -lpp -llwip -lphy -lnet80211 -lwpa -lmain -Wl,--end-group -lgcc -o main /media/lysander/Data/Workspace/ESP8266/ESP8266_NONOS_SDK/lib/libmain.a(app_main.o): In function 'user_uart_wait_tx_fifo_empty': (.irom0.text+0x6e0): undefined reference to 'user_rf_cal_sector_set' /media/lysander/Data/Workspace/ESP8266/ESP8266_NONOS_SDK/lib/libmain.a(app_main.o): In function 'flash_data_check': (.irom0.text+0x84b): undefined reference to 'user_rf_cal_sector_set' /media/lysander/Data/Workspace/ESP8266/ESP8266_NONOS_SDK/lib/libwpa.a(wpa_auth.o): In function 'wpa_receive': (.irom0.text+0x9a0): undefined reference to 'aes_wrap' /media/lysander/Data/Workspace/ESP8266/ESP8266_NONOS_SDK/lib/libwpa.a(wpa_auth.o): In function '__wpa_send_eapol': (.irom0.text+0xd73): undefined reference to 'aes_wrap' collect2: error: ld returned 1 exit status <builtin>: recipe for target 'main' failed make: *** [main] Error 1

I don't really know what to do from here, any help would be appreciated, thanks.

1D TOF measurement with RTS-CTS-NULL-ACK ?

Hello, cnlohr,
Great project!

For surveillance of crop irrigation systems,
I am looking for a 1D-distance in the range up to 350 m, with ~10m accuracy.
I think this is feasible with wlan directional antennae.

From an earlier google scan, I kept this link (in German only):
https://chemnitzer.linux-tage.de/2011/vortraege/folien/653_haustein.pdf
According to them, the RTS-CTS-NULL-ACK sequence is part of IEE 802.11 standard and travels 4 time the transmit line with little jitter.
They use a linux MAC80211 driver, which obviously allows the required manipulations.
They report 1 µs, but by repeating measurements (n >> 1000), they end up at ns resolution.

For my 10m accuracy target, we'd need ~ 100 ns for quadruple path.
Their setup is a master sender/receiver at a fixed location, a slave receiver close to it, and the target receiver at the moving distant target. I think a raspberry or so might be a good master, and esp8266 suitable slaves.

How does this approach fit into the work you have done till now?
Would you like to share efforts?

Wolfgang Rosner

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.