Now you need to run whole things like that:
python3 manage.py crawl "url"
example : python3 manage.py crawl "http://0.0.0.0:3000"

I think we should follow these steps:

• configure crawler
• attack generate
  • list possible url which contain direct reference
  • configure regular expression to capture these direct reference
• attack
  • try different http methods
• analysis response
  • analysis http status code
• report generate

Can you help to build the field's for the action table and result table, {Only store those data in two main table and other's helping data you can store in one or many temp table (if needed)}.

Check on notes file there you will find couple regex pattern, try those and recommend me better regEx to work with.

See the pipelines.py file in that table you need to configure ID (autoincrement) and Link (Unique filed).