Installing and Testing the Critical Stack Intel Client

To help users validate that the Critical Stack Intel Client is working as expected, I wrote a document showing how to identify traffic from Tor exit nodes. Check out the article here

Have more questions? Submit a request

25 Comments

  • 0
    Avatar
    Corey

    Thank you for the guide Richard! One issue arose after installing the Critical Stack Intel Client in that another service (ELSA) on the server no longer functions correctly.

    Would the package install impact MySQL in any way or modify any additional bro .conf file?

  • 0
    Avatar
    Richard Bejtlich

    Hi Corey,

    Thanks for trying the client. I'm using it on my SO system without issues. ELSA works as far as I can tell. Would you mind opening a ticket in Zendesk? You can do it at intel.criticalstack.com, in the upper right side of the page. I think this might be the direct link:

    https://criticalstack.zendesk.com/hc/en-us/requests/new

    Sincerely,

    Richard

  • 0
    Avatar
    Corey

    Thanks Richard. I'll submit a ticket now.

  • 0
    Avatar
    Richard Compton

    Hi Richard, I am having issues installing the Intel Client.
    When I run "sudo critical-stack-intel fetch run" I'm getting the error:

    2015/02/25 16:10:21 --- NOTICE ----------
    2015/02/25 16:10:21 Error: Unable to locate bro or configure permissions properly.
    2015/02/25 16:10:21 Error: Bro not found.
    2015/02/25 16:10:21 If you have a custom setup you can add your paths manually.
    2015/02/25 16:10:21 $ sudo critical-stack-intel config bro.path /my/path/bro
    2015/02/25 16:10:21 $ sudo critical-stack-intel config bro.include.path /my/path/local.bro
    2015/02/25 16:10:21 $ sudo critical-stack-intel config bro.broctl.path /my/path/broctl
    2015/02/25 16:10:21 --- NOTICE ----------

    I have run:
    sudo critical-stack-intel config bro.path /nsm/bro/bin/bro
    sudo critical-stack-intel config bro.include.path /nsm/bro/share/bro/site/local.bro
    sudo critical-stack-intel config bro.broctl.path /nsm/bro/bin/broctl

    but I'm still getting the error message. Is there any documentation on how to configure this or any idea what I am doing wrong? I opened up a Zendesk ticket 3 weeks ago but I haven't heard back from anyone.

    Thanks in advance!

  • 0
    Avatar
    Richard Bejtlich

    Hi RC, we're taking a look and will get back with you. Thank you for trying the client!

    Richard

  • 0
    Avatar
    Richard Compton

    Thank you for the reply!

  • 0
    Avatar
    Sorin Mustaca

    Hi Richard B.,
    I have the same problem as described by RC above.
    Additionally, it seems I have issues to authenticate my API key.

    Strangely, last time i checked it a few weeks ago, everything worked like a charm.
    Now, I have a new system where i test and nothing seems to works anymore.

    Testing on Ubuntu 14.04.

    #sudo critical-stack-intel --debug api <api>
    2015/10/05 18:56:43 This API key has already been added.

    sudo critical-stack-intel pull

    critical-stack 18:56:50 [ERROR] --- NOTICE ----------
    critical-stack 18:56:50 [ERROR] Unable to locate bro or configure permissions properly.
    critical-stack 18:56:50 [ERROR] Unable to add sudoers access for bro binary.
    critical-stack 18:56:50 [INFO] If you have a custom setup you can add your paths manually.
    critical-stack 18:56:50 [INFO] $ sudo critical-stack-intel config --set bro.path=/my/path/bro
    critical-stack 18:56:50 [INFO] $ sudo critical-stack-intel config --set bro.include.path=/my/path/local.bro
    critical-stack 18:56:50 [INFO] $ sudo critical-stack-intel config --set bro.broctl.path=/my/path/broctl
    critical-stack 18:56:50 [ERROR] --- NOTICE ----------
    critical-stack 18:56:50 [INFO] Pulling feed list from the Intel Marketplace.
    critical-stack 18:56:50 [ERROR] Error: Invalid API key.
    critical-stack 18:56:50 [INFO] Please verify the API key you are using is correct.
    critical-stack 18:56:50 [ERROR] Invalid API key

    Thanks
    Sorin

  • 0
    Avatar
    Tim Crothers

    The "apt-get install critical-stack-intel" returns an error no package found.  I triple-checked the prior and "/var/lib/apt/lists/packagecloud.io_criticalstack_critical-stack-intel_raspbian_dists_jessie_main_binary-armhf_Packages" is empty.  Perhaps the raspbian debian package isn't available at the moment?

    In case it's not obvious I'm attempting to build this on a Raspbian Pi 3 running Bro successfully already.

    Thanks!

    Tim

  • 0
    Avatar
    Chris Brown

    Tim, I was having a similar issue.  Here's what worked for me: 

     

    sudo wget --no-check-certificate https://intel.criticalstack.com/client/critical-stack-intel-arm.deb

    sudo dpkg -i critical-stack-intel-arm.deb

    sudo -u critical-stack critical-stack-intel api [API-KEY-HERE]

     

    Hope this helps!

  • 0
    Avatar
    Laurencefield

    Chris, 

    Thank you. The above solved my problems too. 

     

  • 0
    Avatar
    MrPsychoinc4587

    sudo dpkg -i critical-stack-intel-arm.deb

    dpkg: error processing archive crtitical-stack-intel-arm.deb (--install):

    cannot access archive: No such file or directory

    Errors were encountered while processing:

      critical-stack-intel-arm.deb

     

    *Please help*

  • 0
    Avatar
    securid

    For the people running Bro on the Raspberry Pi how do you get your data into Security Onion? I want to run a few Pi's and push all the data to one central spot.

    Thanks

  • 1
    Avatar
    Bo Nilsson

    Hi

    I am building a bro IDS on raspbian. I am running: critical-stack-intel api < MY KEY > and gets this in response:

    critical-stack 08:12:45 [ERROR] Error: operation not supported
    critical-stack 08:12:45 [INFO] Root privileges are required to run critical-stack-intel.
    critical-stack 08:12:45 [INFO] Permissions are dropped when not needed for security.
     I installed and got this:

    root@intrution:/home/pi# dpkg -i critical-stack-intel-arm.deb
    (Reading database ... 38356 files and directories currently installed.)
    Preparing to unpack critical-stack-intel-arm.deb ...
    Unpacking critical-stack-intel (0.5.3) over (0.5.3) ...
    Setting up critical-stack-intel (0.5.3) ...
    Starting Critical Stack Intel ...
    Critical Stack Intel started.
    Add your API key: sudo critical-stack-intel api <key-here>
    Processing triggers for systemd (215-17+deb8u5) ...

    I am bewildered as to why? As you can see I have even been using the root account to install critical-stack after installing bro from source using sudo.

     

    Hope full for assistance,

    bo

  • 0
    Avatar
    Info-france

    @Bo Nilson:

    As you can see after a "cat /etc/passwd" critical-stack is in /bin/false.

    from root => su critical-stack -s /bin/bash :)

    i discovered this today ^^ ! 

  • 0
    Avatar
    Info-france

    by the way ... i currently working on the same project ... Bro + Critical-Stack-Intel + Elasticsearch / logstash / kibana :) 

  • 0
    Avatar
    Wes

    @Info-france

    I'm also working on the same setup as you. 

    Just some questions in the hope you can answer them:
    Did you compiled critical stack intel from source or did you use

         - curl https://packagecloud.io/install/repositories/criticalstack/critical-stack-     intel/script.deb.sh | sudo bash
         - sudo apt-get install critical-stack-intel

    When I tried to apt-get there was no file found on raspbian.
    After that I've compiled it from source but I continuously needed to use 'sudo -u critical-stack' before all commands which is a bit annoying. Besides, I'm not getting the intel.log you supposed to get in the current log folder. The 'normal' bro logs are all working. I've set my api and

    sudo -u critical-stack critical-stack-intel list

    shows my sensors. Could you please tell me the exact commands you used to get critical-stack to work? So from adding the API key onwards. 

     

     

    Edited by Wes
  • 0
    Avatar
    Info-france

    @Wes: 

    Hello ! 

    to install the package: 

    sudo wget --no-check-certificate https://intel.criticalstack.com/client/critical-stack-intel-arm.deb

    sudo dpkg -i critical-stack-intel-arm.deb

    then, check your /etc/passwd.

    if critical stack is store in /bin/false you can switch to the user "critical-stack" from root: su critical-stack -s /bin/bash. From here, just put your api key with the command mentioned above :)

    => critical-stack-intel "api-key".

    then, just fetch your feeds : critical-stack-intel pull

    and, list them: critical-stack-intel list

    You can also edit the /etc/passwd and switch "critical" user to /bin/bash and add it to the sudoers :) 

    Edited by Info-france
  • 0
    Avatar
    Matt

    @Wes @Info-france

    Hi,

    I'm also working on the same setup as you both. I've got Critical Stack integrated with Bro but the issue I'm finding is that not all of the feeds I've subscribed to are downloaded when using 'critical-stack-intel pull'. At the moment only 2 out of 5 test feeds that I've subscribed to are being downloaded to my Pi - are either of you running into the same issue?

    Cheers,

     

    Matt

  • 0
    Avatar
    Wes

    @info-france Thanks for your help!

    So, I finally got it working. Couple of problems I encountered. I hope I can help other people working on the same thing and save them some time. 
    _____________________________________________________________________

    @Mat Yes, I have the same problem, currently those feeds at my setup are:

    • PhishTank-Intel-Feed-(Verified)
    • Scam-Domains-(Fake/Malware/Drive-By)

    Don't know how to fix this. 

    _____________________________________________________________________

    I think the ticketsystem on this site doesn't work.
    When you submit your ticket you can't go back to the page where you're ticket is, and you're supposed to get an email when you've a reaction, but those emails never get delivered as I read other people also never got a response while the team of critical stack claimed they had responded.
    _____________________________________________________________________ 

    After routing all my traffic through the RPi, I thought my critical stack intel wasn't working for a long time. When I had configured everything as it was supposed to be, I used tor on another device to reach certain (normal) sites. But there was no intel.log every time I tried. It made me crazy and I thought I had configured something wrong. But apparently the use of tor in your local network will go unnoticed, only if someone with a tor exit node connects to your local network, it will be noticed. Quite logical, but I hadn't expected it.

    After this, I used tails on a other device, pointed the gateway to the Pi and tried again, this time there was almost no traffic from the tails devices' IP logged in conn.log, even though all the webpages would load. The strangest thing is, just after I enable the internet connection on the tails device I see the tails' IP connecting to a couple external IP's, after that, nothing. Doesn't matter which site I go there is nothing visible in conn.log.

    I'm really curious how this is possible, because I've configured it just like my other devices, from which I do see all traffic in the conn.log. Maybe some recent security measure implemented in tails or something?

    So finally I decided to use a Virtual Machine to connect to some malicious domains, finally there was a intel.log created. So If others experience the same problem, use a VM. 

    _____________________________________________________________________

    I hope someone of the critical stack team knows how to fix the problems above or can give his opinion about this :)

     

    Last but not least,
    check https://github.com/TravisFSmith/SweetSecurity for some handy scripts.

    Edited by Wes
  • 0
    Avatar
    Info-france

    @Wes 

    Same issue ... :'( no more "intel.log", the file disappeared few days after the full install of my setup

    when i will have  a moment, to work on it, i will probably reinstall all the stuff from scratch

  • 0
    Avatar
    psy phii

    Yep since the merger alot of feeds are apparently being blocked by capital ones sophos firewall, judging by the thumbnail of a sophos utm block page that replaced alot of the broken feeds origianl thumbs.

  • 1
    Avatar
    orhiee

    hello, 

    i did a install on a security onion with apt-get and dont really have any of the issues you guys are having 

     

    but i have another issue, cant get all the feeds :( i am having errors as follows:

    critical-stack 21:23:13 [DEBUG] %!(EXTRA string=%s - %s, string=ET: Botnet Command and Control, *errors.errorString=Feed missing: critical-stack-intel-59-ET--Botnet-Command-and-Control.bro.dat, string=)
    critical-stack 21:23:13 [DEBUG] %!(EXTRA string=%s - %s, string=OpenPhish.com (Verified), *errors.errorString=Feed missing: critical-stack-intel-58-OpenPhish.com-(Verified).bro.dat, string=)
    critical-stack 21:23:14 [DEBUG] %!(EXTRA string=%s - %s, string=Ponmocup: Botnet IPs, *errors.errorString=Feed missing: critical-stack-intel-32-Ponmocup--Botnet-IPs.bro.dat, string=)
    critical-stack 21:23:15 [DEBUG] %!(EXTRA string=%s - %s, string=Ponmocup: Malware IPs, *errors.errorString=Feed missing: critical-stack-intel-31-Ponmocup--Malware-IPs.bro.dat, string=)
    critical-stack 21:23:15 [DEBUG] %!(EXTRA string=%s - %s, string=Ponmocup: Botnet Domains, *errors.errorString=Feed missing: critical-stack-intel-30-Ponmocup--Botnet-Domains.bro.dat, string=)

     

    created a different sensor with 3 of the non working feeds (thought it might be not working due to size) but still cant get them 

    did a re-install from the web page - no go 

    any ideas ?

     

    thanks

  • 0
    Avatar
    Raj Kumar

    Hi ,

    Is there any another link  to test it apart from https://csic01.taosecurity.com

    from tails to test critical stack intel.log

     

    Thanks,

    Raj

  • 0
    Avatar
    Ricardo Almeida

    Hi,

    I am having the same problem as @orhiee, but with some other feeds. Output example:

    critical-stack 15:37:40 [DEBUG] %!(EXTRA string=%s - %s, string=hosts-file.net Phishing Domains, *errors.errorString=Feed missing: critical-stack-intel-82-hosts-file.net-Phishing-Domains.bro.dat, string=)
    critical-stack 15:37:40 [DEBUG] %!(EXTRA string=%s - %s, string=ET: Botnet Command and Control, *errors.errorString=Feed missing: critical-stack-intel-59-ET--Botnet-Command-and-Control.bro.dat, string=)
    critical-stack 15:37:41 [DEBUG] %!(EXTRA string=%s - %s, string=OpenPhish.com (Verified), *errors.errorString=Feed missing: critical-stack-intel-58-OpenPhish.com-(Verified).bro.dat, string=)
    critical-stack 15:37:42 [DEBUG] %!(EXTRA string=%s - %s, string=ET: Known Compromised Hosts, *errors.errorString=Feed missing: critical-stack-intel-25-ET--Known-Compromised-Hosts.bro.dat, string=)
    critical-stack 15:37:44 [DEBUG] Downloading file:

    Filename: critical-stack-intel-23-Malware-Domains.bro.dat
    Checksum: 5435c6da6269766ba010d69d6cf6599c

    critical-stack 15:37:45 [DEBUG] %!(EXTRA string=%s - %s, string=PhishTank Intel Feed (Verified), *errors.errorString=Feed missing: critical-stack-intel-18-PhishTank-Intel-Feed-(Verified).bro.dat, string=)

    In this example only the Malware-Domains worked properly.

    Any ideas?

     

     

  • 0
    Avatar
    Karl Hart

    Also having the same issue updating some of the feeds.  Seems that most of the feeds are missing.  Anyone know of a work-around to be able to get the feeds?  

Please sign in to leave a comment.