Brainspace Implementation Guide
Latest Release Announcement
As of February 28, 2022, our latest release is version 6.6 and its subsequent point releases. See our published Release Notes for the latest details on features and bug fixes. This guide is designed to help our clients quickly and easily deploy or upgrade the Brainspace application. If you have any questions, please contact Brainspace Support at support@revealdata.com for assistance.
Brainspace 6.6 Server Guidelines
Server Resource recommendations vary depending on client data, document and dataset size, complexity and the Brainspace features being used.
Use these server recommendations as a guideline only: monitor system resource utilization closely and update as required.
If using VM instances, use Resource Allocation Reservations to ensure committed server performance.
For larger volume instances, please consult your Brainspace Solutions Architect or submit a request to support@revealdata.com.
Additional Sizing Considerations
The provided Brainspace 6.6 Server Guidelines will provide for full Brainspace functionality and performance for a wide range of document populations. Actual resource needs will vary depending on source documents complexity and the combination of Brainspace analytics being used.
Brainspace processes all extracted text and metadata without a requirement for pre-filtering, however large, complex data types can reduce system performance and require additional resources.
Larger datasets tend to consume more RAM than multiple smaller datasets due to larger brains and more complex relationships within the document population.
Server specific resource considerations.
Application Server – All Active datasets are loaded into RAM for optimal graphical response. Large documents and datasets will increase memory requirements. Memory and Processor utilization is driven by Concept Search, Cluster Wheel and Communications Analysis activity.
Analytics Server – Only one dataset can be built at a time on the Analytics server. Additional Analytics servers can be deployed to allow for multiple, parallel dataset builds to occur. Additional resources (RAM & Processors, always at least a 4:1 ratio) may be required for very large dataset builds or may help improve dataset build times.
On-Demand Analytics (ODA) Server – Only one Focus can be built at a time on the ODA server. Additional ODA servers can be deployed to allow for multiple, concurrent Focus builds. Also, machine learning activity for CMML and PC occur on the ODA server and heavy classifier users should increase system resources accordingly. Additional resources (RAM & Processors, always at least a 4:1 ratio) may be required for very large Focus builds or may help improve CMML, PC and Focus build times.
Increased user activity in Brainspace does not drive system resources quite as much as data and system activity. Plan accordingly for dataset volume (number and size of documents) and active projects concurrently in the system. A good rule of thumb for high user counts is to add 0.5GB per active user (i.e. 20GB for 40 concurrently logged in users) to the Application Server.
Disk usage on the /data volume will expand by up to six times the size of your source text during the Brainspace dataset build process, index creation and optimization. Following the dataset completion and cleanup process the data will typically occupy from 1-2x the source text on disk.
Keep in mind that usage requirements will likely change between minor versions (6.1 to 6.2, 6.2 to 6.3, 6.5 to 6.6, etc.) as new features are added.
Disk Layout
Brainspace recommends 100GB of disk storage for each server, utilizing the following layout. Please note that any deviations from this recommendation may impact upgrades or application performance and should be discussed with the Brainspace Support PM or Engineer prior to upgrade or install. Any file storage for client-managed logging or security applications should be in addition to this recommendation.
The /localdata volume required on the Application and On-Demand Analytics servers requires a high-performance storage solution to process large volumes of data for the Brainspace machine learning and advanced graphical analytics features. SSD disks or a High IOPS (≥ 30,000) Enterprise storage solution is recommended, with 10Gbps storage network.
The /data volume must be shared from the Analytics server to all other servers. Brainspace recommends using NFS to share this volume from the Analytics server to the other severs in the solution, however it may also be shared from an Enterprise storage solution with minimum 10,000 IOPS and at least 1Gbps actual throughput between servers.
The /localdata-share (/localdata on App) is the new NFS share from the Application Server to the ODA Server and is a new requirement for 6.6. The NFS must be configured before install, just like /data. During the actual install, the installer asks where the localdata share is, which should be /localdata-share on the On-Demand Server. This is in addition to /localdata and not to replace /localdata.
Information on configuring the /localdata-share can be found here:
This is our recommended disk layout. Alternatives can be configured if desired, so long as the general layout remains as follows:
Mount Point | Volume Size | File System |
|---|---|---|
Application Server | ||
/boot | 250 M | Ext4 |
/var | 50G (' 25G free for upgrade) | Ext4 |
/tmp | 20G | Ext4 |
/ | Remainder | Ext4 |
SWAP | 8GB maximum | SWAP |
/localdata (NFS shared to the ODA server) | Variable | XFS |
/data (NFS shared from Analytics server) | Variable | XFS |
Analytics Server | ||
/boot | 250 M | Ext4 |
/var | 50G (' 25G free for upgrade) | Ext4 |
/tmp | 20G | Ext4 |
/ | Remainder | Ext4 |
SWAP | 1x Server RAM | SWAP |
/data (NFS shared to App & ODA servers) | Variable | XFS |
On Demand Server | ||
/boot | 250 M | Ext4 |
/var | 50G (' 25G free for upgrade) | Ext4 |
/tmp | 20G | Ext4 |
/ | Remainder | Ext4 |
SWAP | 1x Server RAM | SWAP |
/localdata | Variable | XFS |
/localdata-share (NFS shared from App server) | NFS | |
/data (NFS shared from Analytics server) | Variable | XFS |
OS Configuration Requirements
Brainspace 6.6 requires either CentOS or Red Hat Operating System, Version 7.6 with current patches thru 7.9.
The US English version of CentOS or Red Hat should be selected. Other languages may cause data format issues.
Use the ‘Minimal Install’ option to prevent installation of unnecessary packages.
Perform a YUM update to update installed packages to the latest available security and application fixes.
Install the following Linux utilities to your servers for use during installation or subsequent troubleshooting. Your servers will require internet access, or access to a local RPM repository, to install the appropriate packages.
Execute the following command to install the recommended packages:
yum -y install nfs-utils lvm2 sysstat nmap
If your corporate security policy requires other Linux configuration settings, please notify your Brainspace Project Manager, as alternate settings may prevent Brainspace from installing or functioning as required.
Starting in Brainspace 6.6 the perl-libs package is required. In order to install this Perl will need to be installed. This can be installed with the command:
yum -y install perl-libs
Shared Storage Configuration
Brainspace requires a shared file system for the /data volume available to all Brainspace servers. We recommend using a Network File System (NFS) with nolock option enabled to share the /data storage volume hosted from an enterprise network storage (NAS) or from the Analytics server to the other servers in the instance.
For 6.6, we also require that the /localdata on the App server be shared to the ODA Server as /localdata-share.
NFS Configuration Summary /data |
|---|
Analytics Server Export Path: /data |
Application Server Mount Path: /data |
On Demand Analytics Server Mount Path: /data |
NFS Configuration Summary /localdata-share |
|---|
Application Server Export Path: /localdata/brainspace |
On Demand Analytics Server Mount Path: /localdata-share |
Brainspace Network Summary
Important
If using a load balancer or proxy service to access Brainspace, it is recommended that the load balancer be configured to bypass HAProxy, handle any required SSL processing and additional security requirements and connect to the Brainspace Application server on port 8081.
See our HAProxy usage article for more information.
The haproxy service name changed from 6.1 to 6.2 and 6.6 to brainspace-haproxy. Here's a summary of the configuration and changes: Replacing the Default Brainspace Self-Signed SSL Certificates.
Firewall Access Rules Summary
See Appendix B for table of port allowances.
Security Settings
Linux System Accounts
Secure and store your ‘root’ account password in a safe place. Brainspace will not store nor be able to recover this account if lost.
Additional individual accounts should be created for other administrators within your organization, as necessary, to be used for server login, troubleshooting, file copy, etc.
Disable root login (recommended for added security) once you have created the suggested individual login accounts.
The ‘brains’ service account will be created by the Brainspace installer. Do not set a password for this account. You may sudo as the brains user as necessary for troubleshooting.
Brainspace Accounts
A ‘support@revealdata.com’ Super Admin account will be created for your initial Brainspace user interface (UI) access. The Super Admin has the highest level of permissions for completing the installation configuration requirements, as well as for ongoing administrative and troubleshooting tasks across all Groups and Datasets. Brainspace recommends creating individual User, Group Admin and Super Admin accounts for individuals on your team as required to perform their assigned tasks. The original Super Admin account, or other individual account with system-level permissions, will be needed by Brainspace Support personnel should we need to assist your team for future troubleshooting issues. Please do not change the password, or if you do, keep the password in a safe and secure location.
Note
If your corporate security policy conflicts with any of these recommendations, your security policy will take precedent. Please discuss any concerns with your Project Manager to avoid disruption to your application functionality.
SSL Certificates
By default, all Brainspace interfaces (UI and inter-server communications) use self-signed SSL
certificates. You may provide your own domain signed certificate for application access and encryption, if required.
Provide certificate in PEM (Base64-encoded ASCII) format, an unencrypted private key, and any intermediate certificates.
Brainspace can also be configured to offload SSL-encryption to a load balancer or proxy server. Please let your Project Manager or Solutions Architect know if you plan to do SSL encryption on another device.
Scheduling Brainspace 6.6 Installation or Upgrade
Planning
Once your servers are built and the environment configured as required, perform an audit of your systems to verify all prerequisites are completed to avoid delays during, or risk rescheduling, the installation.
Brainspace provides an automated audit script to help audit and verify your servers are ready for the installation or upgrade. The script gathers relevant server specifications (processors, memory, swap, etc.) in a text file for easy review or to submit to Brainspace for review prior to scheduling installation resources. See our Infrastructure Audit article for full audit details: Preinstallation Script: Server Configuration Audit.
Resolve any exceptions noted in the audit results. Brainspace requires a clean system audit before installation resources can be scheduled. If you have questions or are unable to complete any of the configuration requirements on your own, please contact your Project Manager to discuss your questions or to schedule a technical review call with one of our engineers to address any concerns regarding environment build and configuration.
Installation or Upgrade
Brainspace installation resources normally won’t have access to your servers and will rely on a screen sharing application ie; WebEx, Zoom, etc. The screenshare presenter must have full access to the server environment where Brainspace will be installed.
Brainspace application installation or upgrade generally take between 2-4 hours, depending on server access and applicable integration with your third-party applications.
Key Resources Required for Installation
IT (server, storage and network) – Available to assist with server access and to remediate any environment issues that may hinder the Brainspace installation/upgrade process.
Relativity®/Nuix Admins (as applicable) – Access accounts, plus the ability to access test data within Relativity®/Nuix, will be required during the installation. Please make sure you have a test workspace in Relativity®, or a test case in Nuix, to verify functionality after installation.
Brainspace Users/Admins – Following the installation or upgrade, we will perform a brief product orientation, including demonstration on data ingestion and the key application features, to prepare your team to begin immediately realizing the power and new features in Brainspace 6.6. We will also show how to troubleshoot common issues, download relevant logs and submit questions/problems to Brainspace Support when you need assistance. All Brainspace Users and Admins are encouraged to participate.
Brainspace Integration Requirements
Relativity® Integration for versions 9.7+ and 10.x+ (including RelativityOne cloud)
Brainspace has included a new connector as of version 6.2 with improved security, performance and functionality over our legacy connector. The new Relativity® Plus connector requires only Relativity® API access via port 443 (https) and only works with Relativity® versions 9.7 and newer. In Brainspace 6.6 you can now use the Relativity® Connector on datasets originally built with the Relativity® Connector, as long as the “Use Legacy Names” toggle is on.
Relativity® Integration for versions 9.2 – 9.7 (on-prem deployments only)
Brainspace integrates with older versions of Relativity® using our legacy Relativity® connector. This connector requires access to the Relativity® REST API (https) and direct SQL access (tcp 1433) using Brainspace legacy Relativity® connector.
Brainspace’s legacy Relativity® connector uses the RSAPI service to determine user access permissions to assigned workspaces, and MS-SQL access for high-speed ingestion and sync of Brainspace data back to Relativity®. The Brainspace Relativity® plugin respects the Relativity® ACLs (access control list) for data streaming and overlay back to Relativity® – Workspace access and updates will only occur with proper user authentication credentials.
Integrating with a Single Relativity® SQL Instance
Brainspace utilizes TCP port 1433 (by default; please indicate if you are using a non-standard SQL port) to access the Relativity® MS-SQL database for integration.
Integrating with Multiple/Distributed Relativity® SQL Instances
If using Static ports – Configure each SQL instance with the same port (typically 1433).
If using Dynamic ports – The MS SQL Browser Service must be configured (typically port 1434 UDP) for Brainspace to determine the actual port in use for each SQL instance. The full range of dynamic ports in use must be allowed through the firewall from the Brainspace Application server to each Relativity® SQL instance.
The Relativity® MS-SQL instance must be configured and available prior to Brainspace installation. This account must have EDDSDBO user rights. If unable to use EDDSDBO account, the provided account permissions will need to either be added manually or a script written to automatically update the “brainspace” user to all new workspaces.
Server Role = public
For Workspaces to be used with Brainspace, the database user needs db_datareader, db_datawriter, db_ddladmin and roles
For the main EDDS database, the database user needs db_datareader. For enhanced security, a custom role providing read only access to the Case and ResourceServer tables may be used.
For additional security, set the minimum required column access in Relativity® EDDS db:
Case ArtifactId
Case ServerId
ResourceServer ArtifactId
ResourceServer Name
The MS-SQL account must be configured on all SQL instances in a distributed SQL environment.
Nuix Integration
Nuix integration requires the Nuix RESTful Service to be installed and configured on your Nuix Server. Go to Nuix.com to see references for common Nuix versions.
Verify the Nuix RESTful API service is configured and listening by executing a curl command to invoke the Nuix health check from the Brainspace Application (Runtime) server.
If NUIX RESTful API interface is configured for SSL:
curl -k https://nuix servername:8443/nuix-restful-service/svc/v1/system/health
If NUIX RESTful API interface is configured for non-SSL:
curl http:// nuix servername:8080/nuix-restful-service/svc/v1/system/health
Expected Result: "success" : true
Appendix A: Pre-Install Checklist: Server Infrastructure Readiness
Fill out the Pre-Install Checklist to ensure a successful Brainspace installation. Completion is required before installation resources can be scheduled. Contact your Project Manager if you need assistance or have questions with any configuration item. (double-click the checkbox to mark item completed)
Check | Items to Complete | Additional Information |
|---|---|---|
Server Configuration | Confirm server configuration (OS, Cores, RAM, Disks, Swap, DNS, NTP, patching, etc.) meets Brainspace required specifications. | Run automated system audit script and resolve any identified issues. Full Audit instructions |
Upload final audit script results to support@revealdata.com. | ||
| ||
SSL Certificate | Which certificate type will you use? | Select all that apply:
|
Install and License Files available | Copy Brainspace installation files (installer + license) to /data directory on Application server and to root (/) on the On-Demand Analytics server. |
|
Reboot servers | Reboot all servers to ensure services startup properly before beginning the Brainspace installation. Restart the Analytics Server, the Application Server, then the On-Demand Analytics Server to ensure NFS services come online. | 0 Servers rebooted (After NFS configuration & Before Brainspace installation) |
Analyst Readiness | Have all of your users (Analysts and Administrators) received training on Brainspace 6? 0 Yes, our users are ready! 0 Please send me more information on training. | Brainspace offers both self-guided and formalized training opportunities. Self-Paced |
https://help.revealdata.com – Updated User Guides | ||
Brainwaves – Online Admin and Analyst training | ||
Environment Security Standards | FIPS Compliance | 0 Compliance restrictions discussed with PM |
Supported Browsers | Confirm your users have Chrome, Edge (Chromium-based versions) or Firefox installed with WebGL enabled | 0 Users have a supported browser |
Appendix B: Pre-Install Checklist: Network Access
Source | Destination | Ports | Description |
|---|---|---|---|
Brainspace Common Ports | |||
End Users | Application Server | 443 (HTTPS) | User access to User Interface |
System Admin | Application, Analytics, ODA Servers | 22 (SSH) | Admin access to Secure Shell for System Administrative tasks |
Analytics & ODA Servers | Application Server | 8081 (HTTPS/Tomcat) | Inter-server communication to the Application Server |
Application Server | Analytics & ODA Servers | 1604 (HTTPS) | Inter-server communication from the Application Server |
Application Server | Analytics Server | 111 TCP/UDP (RPC) 2049 TCP/UDP (NFS) 20045 TCP/UDP (STATD) 20047 TCP/UDP (LOCKD) 20048 TCP/UDP (MOUNTD) | Network file sharing for /data |
ODA Server | Application Server | 111 TCP/UDP (RPC) 2049 TCP/UDP (NFS) 20045 TCP/UDP (STATD) 20047 TCP/UDP (LOCKD) 20048 TCP/UDP (MOUNTD) | Network file sharing for Application’s /localdata/brainspace to ODA’s /localdata-share |
Application, Analytics, ODA Servers | Application Server, or PostgreSQL Server if combined | 5432 (PostgreSQL) | Communications to the PostgreSQL database |
Relativity® Integration Ports | |||
Application Server | Relativity® RSAPI Server | 443 (HTTPS) | Relativity® REST API access |
Application Server | Relativity® MS SQL Server(s) | 1433 TCP (SQL) 1434 UDP (SQL Browser Service) | Relativity® SQL Access, only if using Relativity® Classic connector |
Relativity® Integration Ports | |||
Application Server | Nuix RESTful API Server | 8080 (HTTP) or 8443 (HTTPS) | Nuix RESTful API access |
Appendix C: Post-Install Checklist (Brainspace Verification)
Complete this checklist after installation to ensure Brainspace is ready for use by your team.
Check | Additional Information | Results |
|---|---|---|
Verify Admin functions | Datasets: Add from DAT file (requires Connector) and view dataset info. | |
Create tags. Download Archive, Process, Status reports; Build Error and Build logs and Import Error Archive. Explain each. Users and Groups: Create ‘customer-super@email.com ( | ||
account. Create test Group, User and Notebook; grant access to test Notebook. Show how User cannot access other Group Notebooks. Connectors: Setup Relativity® or Nuix, DAT connectors | ||
Services: show license status, documents in use, expiration date and | ||
disk and memory usage. Download brainspace.log & catalina.out logs from App, analysisServer.out and analysisserver.log files from Analytics servers. Upload to install ticket. Errors: View latest. Archive. | ||
Create Enron10k dataset from local disk | Verify build completes to ensure functionality. Performance wise, we expect a full build to complete in 6 to 7 minutes. | |
Build Enron10k from DAT | Perform build from DAT completes without errors | |
Concept Search & Cluster Wheel | Search for ‘birthday’ and verify results on Cluster Wheel. Top Terms (time, day, good, work, mom, great, etc.) and Top Concepts (happy birthday, birthday party, celebrate, cake, tortillas, dinosaur, annie, brisket, etc.). Add block of Terms from Cluster Wheel, verify further filtering on Dashboard. | |
Create a Notebook | Create a Notebook from search results. | |
Add to a Notebook | Remove Duplicate documents. Add search results to prior saved Notebook. | |
Create a Focus | Remove block of Terms filter, use results to Create Focus. | |
Communication Analysis | Select 2 people while holding Shift key. Verify people are highlighted and results display for Sent (unique docs, To, cc, bcc, etc.), Received, Sends Most To, Receives Most From, etc. | |
Near Duplicates | Verify ND documents at Cluster Wheel edge. Add to search criteria. | |
License Verification & Usage Reporting | Verify license usage and expiration details (Admin\Services\Application). | |
Download and send to Brainspace (reports@revealdata.com) | ||
Remove Unused Plug-Ins (Brainspace Only) | Navigate to /var/lib/brains/.brainspace/plugins and remove applicable, unused plug-ins | |
Date Format | Show AUTO (US ISO) and EURO Date Format plus the ability to custom configure any consistent date format. | |
Capture Version number | Access the server details under the Services tab |
Appendix D: Post-Install Checklist (Relativity® / Nuix)
We will complete this checklist after installation to ensure Brainspace is ready for use by your team.
Relativity® Checks (if applicable) | Description | Additional Information | Results | |
|---|---|---|---|---|
1 | Relativity® applications | Verify install | Confirm the Brainspace Addons Application is installed to at least one workspace. | |
2 | Relativity® Connector Set Up | Add database details (URL, ports, password, etc) and Fields | Confirm after setting up connector. Select ALL overlay fields. | |
3 | Dataset | Create Dataset via Ingest Wizard | Apply customer supplied credentials, create the following Field mappings: *DOCID, *Fulltext (bodytext), Date, DateSent, To, From, CC, BCC. Facet To, From, CC, Date and BCC. (*Minimum required fields) | |
4 | Dataset | Submit Dataset | During submission, tail brainspace.log and info.log to watch for any errors | |
5 | Sync | Sync Notebook | Confirm functionality via results returned (e.g. has images) | |
6 | Sync | Relativity® Overlay (Test in Relativity®) | Automatic overlay test via executing ‘BD all coding fields’ saved search. Confirm EMT fields populated. | |
7 | Validate | Record Count | # of records and time for build | |
8 | Sync | Application Library | Push Brainspace application from Test workspace to Relativity® Application Library and unlock (push to library) | |
Nuix checks (if applicable) | ||||
1 | Dataset | Create Dataset | Create Dataset from Nuix test case | |
2 | Export | Initiate Export | Verify Export | |
3 | Sync | Test sync | Tag documents and sync to Nuix | |
Appendix E: Start-Up Order for Servers and Services
Use the following guidelines for server and services start-up:
If you are using NFS to share the /data volume from the Analytics server to other servers in your environment, the NFS service on the Analytics server must be started before NFS client instances. This typically means you would start your Analytics server first, and all required services should start automatically (especially NFS, analysis server).
Start the Postgres database next if Postgres is deployed on a dedicated server (not on the Application server).
If Postgres is installed on the Application server, start the Application server, and all required services should start automatically (especially Postgres, NFS client, and Tomcat).
Start the On-Demand Analytics server next, and all required services should start automatically (especially NFS and analysis server).
If you have multiple, redundant Analytics or On-Demand Analytics servers, those should be started at this point.