Update the Bazel rules to support Bazel 4.0.0 (#10)

Updates the Bazel code and dependencies so that it can be successfully built and run on Bazel 4.0.0. Note that this is breaking, the newer Bazel dependencies will throw an error if you try to run it on Bazel 1.1.0 (the previous tested version).

13 files changed, 47 insertions, 62 deletions

diff --git a/.bazelrc b/.bazelrc
index 5ca1659..31f2035 100644
--- a/.bazelrc
+++ b/.bazelrc
@@ -15,3 +15,6 @@ build \
 # build --action_env=LD_PRELOAD
 # test --action_env=LD_PRELOAD
 # run --action_env=LD_PRELOAD
+
+# Suggested here: https://github.com/bazelbuild/rules_foreign_cc/issues/528
+build --action_env='MAKEFLAGS="-j$(expr $(nproc) / 4 '|' 1) "'
diff --git a/.github/workflows/test_syrenn.yml b/.github/workflows/test_syrenn.yml
index ed0d668..ed271f0 100644
--- a/.github/workflows/test_syrenn.yml
+++ b/.github/workflows/test_syrenn.yml
@@ -5,12 +5,12 @@ on: [push]
 
 jobs:
   test-server:
-    runs-on: ubuntu-16.04
+    runs-on: ubuntu-20.04
     steps:
     - uses: actions/checkout@v1
     - name: Build and test the C++ SyReNN server.
       run: |
-        V=1.1.0
+        V=4.0.0
         OS=linux
         ARCH=x86_64
         GH_BASE="https://github.com/bazelbuild/bazel/releases/download/$V"
@@ -23,14 +23,17 @@ jobs:
         chmod +x install.sh
         ./install.sh --user
         rm -f install.sh
-        /home/runner/bin/bazel test syrenn_server/...
+        /home/runner/bin/bazel test syrenn_server/... \
+            --define BAZEL_PYTHON_ONLY_WARN=true \
+            --action_env='MAKEFLAGS="-j2 "' \
+            -j 2
   test-client:
-    runs-on: ubuntu-16.04
+    runs-on: ubuntu-20.04
     steps:
     - uses: actions/checkout@v1
     - name: Build and test the Python SyReNN frontend (PySyReNN), skipping one resource-intensive test and a few that rely on the server to be running.
       run: |
-        V=1.1.0
+        V=4.0.0
         OS=linux
         ARCH=x86_64
         GH_BASE="https://github.com/bazelbuild/bazel/releases/download/$V"
diff --git a/Makefile b/Makefile
index 803bdb1..a68feb6 100644
--- a/Makefile
+++ b/Makefile
@@ -9,7 +9,7 @@ pysyrenn_coverage:
 	bazel run coverage_report
 
 start_server:
-	bazel run syrenn_server:server
+	bazel run syrenn_server:server --define BAZEL_PYTHON_ONLY_WARN=true
 
 # Experiments from [1]
 acas_lines_experiment:
diff --git a/README.md b/README.md
index 09f4be5..6e76570 100644
--- a/README.md
+++ b/README.md
@@ -66,17 +66,14 @@ limit use to only the first 16 cores as shown in ``.bazelrc``.
 
 Note that because we compile (almost) everything from source, the first build
 *will likely take a long time*; at least a few minutes. Subsequent builds
-should be much faster. If you have multiple processing cores on your machine
-and would like to use them to speed up the build, you can modify the ``make``
-commands in ``external/*.BUILD`` to ``make -j`` followed by the number of
-threads you would like Make to run. However, because Bazel will simultaneously
-be running other builds processes, too much use of parallelism in the ``make``
-calls can sometimes cause freezing and running out of memory. If your computer
-is already freezing/running out of memory, consider removing the existing
-``-j#`` flags from ``external/*.BUILD``.
+should be much faster. By default, each `make` initiated by Bazel will try to use
+`$(nproc) / 4` processors available on your machine. However, Bazel will also
+parallelize the build, so this could result in a multiple of that number of
+threads. ***If you find your machine hanging or crashing,*** modify `.bazelrc`
+to specify a lower number of cores.
 
 #### Local Builds
-1. You must install [Bazel](https://bazel.build/) 1.1.0 and have binaries for
+1. You must install [Bazel](https://bazel.build/) 4.0.0 and have binaries for
    building arbitrary C++ packages (eg. ``build-essential`` for Ubuntu).
 2. Furthermore, the ``libcairo2``, ``libffi-dev``, ``zlib1g-dev``, ``zip``, and
    ``libgmp3-dev`` packages are required for the Python code (but usually come
@@ -98,7 +95,7 @@ is already freezing/running out of memory, consider removing the existing
 Alternatively, a Docker container is provided to simplify the build and running
 process. To use it, first build the image with ``./docker_build.sh`` then
 prepend ``./docker_run.sh`` to all of the commands below. For example, instead
-of ``make start_server``, use ``./docker_run.sh make start_server``. Everything
+of `make start_server`, use `./docker_run.sh make start_server`. Everything
 should be handled transparently.
 
 **NOTE:** Benchexec is currently not supported under the Docker container due
diff --git a/WORKSPACE b/WORKSPACE
index e48cbef..bad0d9f 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -5,7 +5,7 @@ load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
 
 git_repository(
     name = "bazel_python",
-    commit = "f99ab8738dced7257c97dc719457f50a601ed84c",
+    commit = "cb4f346641afd090dc088c01df46dc403e600773",
     remote = "https://github.com/95616ARG/bazel_python.git",
 )
 
@@ -19,14 +19,14 @@ bazel_python()
 all_content = """filegroup(name = "all", srcs = glob(["**"]), visibility = ["//visibility:public"])"""
 
 # Rule repository
-http_archive(
+git_repository(
     name = "rules_foreign_cc",
-    sha256 = "82811053dbeb6cefabc9e69c6775ae45d4a13ebfc7c301cc3f6df3eb40947e99",
-    strip_prefix = "rules_foreign_cc-adb04eed2c058b4b161b321b1104eb55ef63b0f4",
-    url = "https://github.com/bazelbuild/rules_foreign_cc/archive/adb04eed2c058b4b161b321b1104eb55ef63b0f4.zip",
+    commit = "b8b88cd2d16035aa1639434eb808f4d67a34d5ae",
+    remote = "https://github.com/bazelbuild/rules_foreign_cc.git",
+    shallow_since = "1620401997 -0700",
 )
 
-load("@rules_foreign_cc//:workspace_definitions.bzl", "rules_foreign_cc_dependencies")
+load("@rules_foreign_cc//foreign_cc:repositories.bzl", "rules_foreign_cc_dependencies")
 
 rules_foreign_cc_dependencies()
 
@@ -75,28 +75,20 @@ http_archive(
 
 ##### BEGIN GRPC #####
 git_repository(
-    name = "upb",
-    commit = "10a18cc12c2ebbcc0ed3aac1e61ae75d9bfc69d8",
-    remote = "https://github.com/matthewsot/upb.git",
-)
-
-load("@upb//bazel:workspace_deps.bzl", "upb_deps")
-
-upb_deps()
-
-git_repository(
     name = "com_github_grpc_grpc",
-    commit = "334c826e8c1a7283c822cd3f11cda1dddde734d7",
+    commit = "8664c8334c05d322fbbdfb9e3b24601a23e9363c",
     remote = "https://github.com/grpc/grpc.git",
+    shallow_since = "1619560885 -0700",
 )
 
 load("@com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")
 
 grpc_deps()
 
-load("@build_bazel_rules_apple//apple:repositories.bzl", "apple_rules_dependencies")
+# This is annoying and bloated because it assumes we want, e.g., Go support.
+load("@com_github_grpc_grpc//bazel:grpc_extra_deps.bzl", "grpc_extra_deps")
 
-apple_rules_dependencies()
+grpc_extra_deps()
 
 ##### END GRPC #####
 
diff --git a/external/eigen.BUILD b/external/eigen.BUILD
index 305b4c6..bb5116d 100644
--- a/external/eigen.BUILD
+++ b/external/eigen.BUILD
@@ -3,9 +3,9 @@ filegroup(
     srcs = glob(["**"]),
 )
 
-load("@rules_foreign_cc//tools/build_defs:cmake.bzl", "cmake_external")
+load("@rules_foreign_cc//foreign_cc:defs.bzl", "cmake")
 
-cmake_external(
+cmake(
     name = "eigen",
     # These options help CMake to find prebuilt OpenBLAS, which will be copied into
     # $EXT_BUILD_DEPS/openblas by the cmake_external script
@@ -13,12 +13,8 @@ cmake_external(
         "BLAS_VENDOR": "OpenBLAS",
         "BLAS_LIBRARIES": "$EXT_BUILD_DEPS/openblas/lib/libopenblas.a",
     },
-    headers_only = True,
     lib_source = "all",
-    make_commands = [
-        "make",
-        "make install",
-    ],
+    out_headers_only = True,
     visibility = ["//visibility:public"],
     # Dependency on other cmake_external rule; can also depend on cc_import, cc_library rules
     deps = ["@openblas"],
diff --git a/external/mkldnn.BUILD b/external/mkldnn.BUILD
index 0a155cb..1c92806 100644
--- a/external/mkldnn.BUILD
+++ b/external/mkldnn.BUILD
@@ -3,10 +3,10 @@ filegroup(
     srcs = glob(["**"]),
 )
 
-load("@rules_foreign_cc//tools/build_defs:cmake.bzl", "cmake_external")
+load("@rules_foreign_cc//foreign_cc:defs.bzl", "cmake")
 
 # MKL-DNN for conv2d
-cmake_external(
+cmake(
     name = "mkldnn",
     cache_entries = {
         # MKL-DNN's source throws set-but-not-used warnings with -Werror,
@@ -16,12 +16,11 @@ cmake_external(
         "WITH_EXAMPLE": "OFF",
     },
     lib_source = "all",
-    make_commands = [
-        "make -j8",
-        "make install",
-        "ls",
+    out_shared_libs = [
+        "libmkldnn.so.1.0",
+        "libmkldnn.so.1",
+        "libmkldnn.so",
     ],
-    shared_libraries = ["libmkldnn.so.1"],
     visibility = ["//visibility:public"],
     alwayslink = True,
 )
diff --git a/external/openblas.BUILD b/external/openblas.BUILD
index a58ca40..0f51d9f 100644
--- a/external/openblas.BUILD
+++ b/external/openblas.BUILD
@@ -3,9 +3,9 @@ filegroup(
     srcs = glob(["**"]),
 )
 
-load("@rules_foreign_cc//tools/build_defs:cmake.bzl", "cmake_external")
+load("@rules_foreign_cc//foreign_cc:defs.bzl", "cmake")
 
-cmake_external(
+cmake(
     name = "openblas",
     # Values to be passed as -Dkey=value on the CMake command line;
     # here are serving to provide some CMake script configuration options
@@ -16,16 +16,11 @@ cmake_external(
     },
     lib_source = "all",
     linkopts = ["-lpthread"],
-    make_commands = [
-        "make -j4",
-        "make install",
-    ],
-
     # We are selecting the resulting static library to be passed in C/C++ provider
     # as the result of the build;
     # However, the cmake_external dependants could use other artefacts provided by the build,
     # according to their CMake script
-    static_libraries = ["libopenblas.a"],
+    out_static_libs = ["libopenblas.a"],
     visibility = ["//visibility:public"],
     alwayslink = True,
 )
diff --git a/third_party/README.md b/third_party/README.md
index 0ab7e8d..f90f130 100644
--- a/third_party/README.md
+++ b/third_party/README.md
@@ -2,4 +2,4 @@
 This directory contains experiments using primarily 3rd-party code, eg.
 bounded-model-checking with ReluPlex and precondition-computation with ERAN.
 They are packaged as Docker containers which can be run using Bazel scripts from
-the base of this repository.
\ No newline at end of file
+the base of this repository.
diff --git a/third_party/eran_bmc/README.md b/third_party/eran_bmc/README.md
index efc052b..d595bd0 100644
--- a/third_party/eran_bmc/README.md
+++ b/third_party/eran_bmc/README.md
@@ -7,7 +7,7 @@ We currently only test the [DeepPoly](https://doi.org/10.1145/3290354) domain
 on the Pendulum model. Other domains may be implemented in the future.
 
 # Building & Running
-1. Install Bazel 1.1.0, Docker (tested with version 18.09.6, build 481bc77),
+1. Install Bazel 4.0.0, Docker (tested with version 18.09.6, build 481bc77),
    and any prerequisites mentioned in [../../README.md](../../README.md).
 2. Ensure that the ``runexec`` line in [../../.bazelrc](../../.bazelrc) is
    commented out (we run ``runexec`` directly in the Docker containers).
diff --git a/third_party/eran_preconditions/README.md b/third_party/eran_preconditions/README.md
index 53465d6..9edbf4a 100644
--- a/third_party/eran_preconditions/README.md
+++ b/third_party/eran_preconditions/README.md
@@ -8,7 +8,7 @@ domain as it provides the necessary representation for precondition
 computation. Other domains may be implemented in the future.
 
 # Building & Running
-1. Install Bazel 1.1.0, Docker (tested with version 18.09.6, build 481bc77),
+1. Install Bazel 4.0.0, Docker (tested with version 18.09.6, build 481bc77),
    and any prerequisites mentioned in [../../README.md](../../README.md).
 2. Ensure that the ``runexec`` line in [../../.bazelrc](../../.bazelrc) is
    commented out (we run ``runexec`` directly in the Docker containers).
diff --git a/third_party/marabou_model_checking/README.md b/third_party/marabou_model_checking/README.md
index 9d0123b..452af4e 100644
--- a/third_party/marabou_model_checking/README.md
+++ b/third_party/marabou_model_checking/README.md
@@ -3,7 +3,7 @@ Code to use Marabou for bounded-model-checking. This is the Marabou
 counter-part to [../reluplex_model_checking](../reluplex_model_checking).
 
 # Building & Running
-1. Install Bazel 1.1.0, Docker (tested with version 18.09.6, build 481bc77),
+1. Install Bazel 4.0.0, Docker (tested with version 18.09.6, build 481bc77),
    and any prerequisites mentioned in [../../README.md](../../README.md).
 2. Ensure that the ``runexec`` line in [../../.bazelrc](../../.bazelrc) is
    commented out (we run ``runexec`` directly in the Docker containers).
diff --git a/third_party/reluplex_model_checking/README.md b/third_party/reluplex_model_checking/README.md
index fb201de..f9ea6d3 100644
--- a/third_party/reluplex_model_checking/README.md
+++ b/third_party/reluplex_model_checking/README.md
@@ -4,7 +4,7 @@ counter-part to
 [../../experiments/model_checking.py](../../experiments/model_checking.py).
 
 # Building & Running
-1. Install Bazel 1.1.0, Docker (tested with version 18.09.6, build 481bc77),
+1. Install Bazel 4.0.0, Docker (tested with version 18.09.6, build 481bc77),
    and any prerequisites mentioned in [../../README.md](../../README.md).
 2. Ensure that the ``runexec`` line in [../../.bazelrc](../../.bazelrc) is
    commented out (we run ``runexec`` directly in the Docker containers).