Syslog log collector connected but no data about cloudappsecuritydocs HOT 18 CLOSED

Hello there!
Just an update in case anyone else is struggling with this. I went into our SonicWall and the only difference I noticed in our logs and the sample was the UTC timestamp. After checking the box in the SonicWall to use UTC timestamps in logs, we now have successful log parsing!

from cloudappsecuritydocs.

I too am dealing with what seems to be this exact issue. I had a fully functional environment up until a few weeks ago and now I can't get anything to work. I'm not entirely convinced that Dell's recent SonicWALL update isn't playing a role in this either.

@joshie45 I have tried your suggested fix on the Docker command -p 514-515:514-515/UDP in my case and it did not make a difference. I have two sources of data that are supposed to be collected by my caslogcollector (Meraki WAPs on UDP 514 and SonicWall on UDP 515). In regards to the Meraki WAPs, I see the traffic being passed to the caslogcollector when I look at Azure logs, but it doesn't seem to be doing anything with it. In regards to the SonicWall traffic, I don't believe it's even attempting (tried to verify using the following method here) to send it to the caslogcollector anymore. I did see it sending them to another log analyzer that has nothing to do with this. We're running SonicOS Enhanced 6.5.1.2-52n on our firewall.

Sorry, there's tons of info in here and possibly multiple issues. Happy to check on anything that you have questions about. In short, I think there is an issue with the configuration of the docker log collector AND I think there is an issue with the latest version of SonicOS.

from cloudappsecuritydocs.

Hi! Great discussion. Please feel free to reach out to our CAS team directly so our developers and product team can help provide insight. [email protected]

from cloudappsecuritydocs.

I was also with problems related to Log format.

I can confirm successful parse on Governance Log with firmware 6.5.1.3-11n.
I used the configuration provided by Cloud App Portal for the Docker. I used the Firewall IP on the configuration (The LAN IP of the sonicwall that was on the same network as the collector).

I Changed to Enhanced format with all options On.
And finally i activated the UTC option and things started to work.

from cloudappsecuritydocs.

@memsim1010,
Setting SYSLOG=true has not worked for me. I was wondering whether or not we need to add "-p 514-600:514-600" to the docker run command as well?

@rkarlin does this seem right to you?

from cloudappsecuritydocs.

Hi,
You were right in setting the Syslog flag to true. Please contact support directly to work through this issue.
Thanks!

from cloudappsecuritydocs.

@rkarlin I made the -e "SYSLOG=true" change in conjunction with the -p 514-515:514-515/UDP addition and it's working. I'm not sure whether or not my change was necessary, but I'm too afraid to redo it an find out. Thanks.

from cloudappsecuritydocs.

@philkloose I forgot the /UDP at the end of the ports, so I will try that again and advise. Hopefully it helps new people in the process :)

from cloudappsecuritydocs.

Making the following changes worked for me also; -e "SYSLOG=true" in conjunction with -p 514-515:514-515/UDP now gives me "Data is being successfully uploaded from all linked data sources." in Cloud App Security. Thanks for your help @philkloose

from cloudappsecuritydocs.

@philkloose I'm trying to set this up for nearly a week now and only get "Connected" in the CAS Portal Log Collector. My Sonicwall is using Firmware 6.5.1.1-42n and don't get it to work.
I'm still confused about the IP address in the creation dialog of the LogCollector? What IP address have you put in there? The IP of the Ubuntu Docker machine where the LogCollector is running on or the internal Sonicwall IP address (what the Microsoft Support told me yesterday). Form the screenshot I cannot figure out which is the correct one as there are everywhere different IP addresses of examples which do not relate to each other.

Best regards,
Thomas

from cloudappsecuritydocs.

from cloudappsecuritydocs.

from cloudappsecuritydocs.

@TLinnepe The IP address is asking for is the one that your log collector will be collecting syslogs from other network devices on. In my case, they come over a VPN so it's a private IP address. In your case it sounds like it would be the "IP of the Ubuntu Docker machine where the LogCollector" is. It needs to know this so that it can tailor the auto-generated CAS log collector commands to your environment.

@memsim1010 A couple of thoughts --

On the Sonicwall device

• verify that you've selected the Enhanced Syslog format
• verify that you've selected everything in the Enhanced Syslog Fields Settings section

In the CAS Portal

• verify that you're sending to the correct port if you have multiple data sources configured. For example, I have two and I have to make sure that my Meraki WAPs send to 514 and the Sonicwall goes to a non-standard 515. If you get those crossed, it won't have any idea how to interpret the data.

I've had a lot of issues with CAS lately. Even after setting it up successfully, it seems to stop ingesting data for no particular reason after a few hours. Rerunning the auto-generated docker commands will bring it back to life, but I'm not sure there isn't something else going on. I have an Azure support case open because their NSG logging seems to be incomplete and it's making troubleshooting very difficult. My NSG flow logs show no evidence of traffic being sent to my caslogcollector VM even though the CAS portal happily reports that it's receiving logs from time to time. It's a mess.

from cloudappsecuritydocs.

from cloudappsecuritydocs.

@memsim1010 Mine is also set to Local Use 0, but like you I'm not sure how important that is. I'm assuming that "SONICWALL_SYSLOG" is the friendly name of the data source. If that's the case, make sure that you also set the source type to Dell SonicWall when you create the source. I'm fairly certain you did this but I can't tell from your initial screenshot and it's worth checking.

Are you also sending your Sonicwall logs to another server unrelated to CAS? I'm not convinced the Sonicwall device is doing exactly what it claims to be doing and in my case I'm wondering it it's due in part to the fact that I'm sending my syslogs to multiple destinations. For whatever it's worth, I feel your pain. This whole process is very brittle. I'm afraid to breathe on it whenever I get it working.

from cloudappsecuritydocs.

@rkarlin Thank you very much. I would like to contact the team about some details.

The sonicwall's enhanced syslog format with all fields check worked for me.

In addition I got it to work last weekend by updating the sonicwalls to Firmware Version 6.5.1.2-52n. I also rebooted the Ubuntu Docker machine but I don't think that this has an affect to the Log uploading.

Edited 18.09.2018:
In the Portal page of the Logcollector creation you must enter the IP address which the firewall is sending in the SYSLog field fw="". In my case it is the public WAN IP address of the Sonicwall.

Later on I installed a local Ubuntu Server 16.04 in my Hyper-V Cluster. Make sure to NOT select automatic updates. This will break the docker instance from running! Also currently I have the issue that logs are only being uploaded when the docker instance is being restarted.

In contact with the Microsoft Support I am now investigation what causes the issue. I will keep you updated on this.

I will set up a blog site with a detailed guide how to set it up this week because there currently is very few content about CAS on the web.

Best regards,
Thomas

from cloudappsecuritydocs.

HI, I'm having a connected with no data issues also. I'm getting this error : No transactions to recognized cloud apps were found while parsing 73 KB from PALO_ALTO_SYSLOG. 200+ logs have been uploaded but i dont see any data in my discovery reports

from cloudappsecuritydocs.

Hi, Thanks for getting in touch. I recommend that you contact support with your issues. This forum is a great place to let me know if there are issues with inaccurate documentation or typos, but if there are problems with the log collector itself, you really should contact your support representative directly.
thanks!

from cloudappsecuritydocs.