Change in bazel[master]: Flag to import external repositories in python import path

1 view
Skip to first unread message

Damien Martin-guillerez (Gerrit)

unread,
Mar 6, 2017, 6:49:47 AM3/6/17
to bazel-de...@googlegroups.com

Damien Martin-guillerez has uploaded this change for review.

View Change

Flag to import external repositories in python import path

This flag will be used to turn off the feature until we
get support for --incompatible flag.

This flag is going to go away very fast, do not rely on it too much.

To be cherry-picked for 0.4.5 (#2472)

Change-Id: I2d3c79ae0c2c53089677573cffd40fa07e03c7e1
---
M src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonConfiguration.java
M src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonSemantics.java
M src/main/java/com/google/devtools/build/lib/bazel/rules/python/stub_template.txt
3 files changed, 24 insertions(+), 6 deletions(-)

diff --git a/src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonConfiguration.java b/src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonConfiguration.java
index fa5e219..d1768e1 100644
--- a/src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonConfiguration.java
+++ b/src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonConfiguration.java
@@ -48,6 +48,12 @@
       category = "version",
       help = "Local path to the Python3 executable.")
     public String python3Path;
+
+    @Option(name = "experimental_python_import_all_repositories",
+      defaultValue = "true",
+      category = "undocumented",
+      help = "Do not use.")
+    public boolean experimentalPythonImportAllRepositories;
   }
 
   /**
@@ -85,4 +91,8 @@
   public String getPython3Path() {
     return options.python3Path;
   }
+
+  public boolean getImportAllRepositories() {
+    return options.experimentalPythonImportAllRepositories;
+  }
 }
diff --git a/src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonSemantics.java b/src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonSemantics.java
index 06fe867..a913927 100644
--- a/src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonSemantics.java
+++ b/src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonSemantics.java
@@ -144,7 +144,9 @@
                   Substitution.of("%python_binary%", pythonBinary),
                   Substitution.of("%imports%", Joiner.on(":").join(imports)),
                   Substitution.of("%workspace_name%", ruleContext.getWorkspaceName()),
-                  Substitution.of("%is_zipfile%", "False")),
+                  Substitution.of("%is_zipfile%", "False"),
+                  Substitution.of("%import_all%",
+                      config.getImportAllRepositories() ? "True" : "False")),
               true));
     } else {
       Artifact zipFile = getPythonZipArtifact(ruleContext, executable);
@@ -160,7 +162,9 @@
                   Substitution.of("%python_binary%", pythonBinary),
                   Substitution.of("%imports%", Joiner.on(":").join(imports)),
                   Substitution.of("%workspace_name%", ruleContext.getWorkspaceName()),
-                  Substitution.of("%is_zipfile%", "True")),
+                  Substitution.of("%is_zipfile%", "True"),
+                  Substitution.of("%import_all%",
+                      config.getImportAllRepositories() ? "True" : "False")),
               true));
 
       ruleContext.registerAction(
diff --git a/src/main/java/com/google/devtools/build/lib/bazel/rules/python/stub_template.txt b/src/main/java/com/google/devtools/build/lib/bazel/rules/python/stub_template.txt
index 00ebcc5..0e28ce1 100644
--- a/src/main/java/com/google/devtools/build/lib/bazel/rules/python/stub_template.txt
+++ b/src/main/java/com/google/devtools/build/lib/bazel/rules/python/stub_template.txt
@@ -99,6 +99,13 @@
   zf.extractall(temp_dir)
   return os.path.join(temp_dir, ZIP_RUNFILES_DIRECTORY_NAME)
 
+# Returns repository roots to add to the import path.
+def GetRepositoriesImports(module_space, import_all):
+  if import_all:
+    repo_dirs = [os.path.join(module_space, d) for d in os.listdir(module_space)]
+    return [d for d in repo_dirs if os.path.isdir(d)]
+  return [os.path.join(module_space, "%workspace_name%")]
+
 def Main():
   args = sys.argv[1:]
 
@@ -111,10 +118,7 @@
 
   python_imports = '%imports%'
   python_path_entries = CreatePythonPathEntries(python_imports, module_space)
-
-  repo_dirs = [os.path.join(module_space, d) for d in os.listdir(module_space)]
-  repositories = [d for d in repo_dirs if os.path.isdir(d)]
-  python_path_entries += repositories
+  python_path_entries += GetRepositoriesImports(module_space, %import_all%)
 
   old_python_path = os.environ.get('PYTHONPATH')
   python_path = os.pathsep.join(python_path_entries)

To view, visit change 9210. To unsubscribe, visit settings.

Gerrit-Project: bazel
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2d3c79ae0c2c53089677573cffd40fa07e03c7e1
Gerrit-Change-Number: 9210
Gerrit-PatchSet: 1
Gerrit-Owner: Damien Martin-guillerez <dmar...@google.com>

Damien Martin-guillerez (Gerrit)

unread,
Mar 6, 2017, 6:52:55 AM3/6/17
to Lukács T. Berki

Damien Martin-guillerez posted comments on this change.

View Change

Patch set 1:

Ok I made the two other changes for the roll out (not ready for review though, just as a reminder for me, I put reminder for them in the following weeks).

I confirmed that with the flag it passes all the test for magenta.

    To view, visit change 9210. To unsubscribe, visit settings.

    Gerrit-Project: bazel
    Gerrit-Branch: master
    Gerrit-MessageType: comment
    Gerrit-Change-Id: I2d3c79ae0c2c53089677573cffd40fa07e03c7e1
    Gerrit-Change-Number: 9210
    Gerrit-PatchSet: 1
    Gerrit-Owner: Damien Martin-guillerez <dmar...@google.com>
    Gerrit-Reviewer: Damien Martin-guillerez <dmar...@google.com>
    Gerrit-Reviewer: Lukács T. Berki <lbe...@google.com>
    Gerrit-Comment-Date: Mon, 06 Mar 2017 11:52:53 +0000
    Gerrit-HasComments: No

    Lukács T. Berki (Gerrit)

    unread,
    Mar 6, 2017, 8:27:33 AM3/6/17
    to Damien Martin-guillerez

    Lukács T. Berki posted comments on this change.

    View Change

    Patch set 1:Code-Review +2

    Filed https://github.com/bazelbuild/bazel/issues/2636 for the rest of the work.

      To view, visit change 9210. To unsubscribe, visit settings.

      Gerrit-Project: bazel
      Gerrit-Branch: master
      Gerrit-MessageType: comment
      Gerrit-Change-Id: I2d3c79ae0c2c53089677573cffd40fa07e03c7e1
      Gerrit-Change-Number: 9210
      Gerrit-PatchSet: 1
      Gerrit-Owner: Damien Martin-guillerez <dmar...@google.com>
      Gerrit-Reviewer: Damien Martin-guillerez <dmar...@google.com>
      Gerrit-Reviewer: Lukács T. Berki <lbe...@google.com>
      Gerrit-Comment-Date: Mon, 06 Mar 2017 13:27:30 +0000
      Gerrit-HasComments: No

      Vladimir Moskva (Gerrit)

      unread,
      Mar 6, 2017, 9:45:58 AM3/6/17
      to Damien Martin-guillerez, Lukács T. Berki

      Vladimir Moskva merged this change.

      View Change

      Approvals: Lukács T. Berki: Looks good to me, approved
      Flag to import external repositories in python import path 
      
      This flag will be used to turn off the feature until we
      get support for --incompatible flag.
      
      This flag is going to go away very fast, do not rely on it too much.
      
      To be cherry-picked for 0.4.5 (#2472)
      
      --
      Change-Id: I2d3c79ae0c2c53089677573cffd40fa07e03c7e1
      Reviewed-on: https://cr.bazel.build/9210
      PiperOrigin-RevId: 149291628
      MOS_MIGRATED_REVID=149291628
      ---
      M src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonConfiguration.java
      M src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonSemantics.java
      M src/main/java/com/google/devtools/build/lib/bazel/rules/python/stub_template.txt
      3 files changed, 24 insertions(+), 6 deletions(-)
      
      
      Gerrit-MessageType: merged
      Gerrit-Change-Id: I2d3c79ae0c2c53089677573cffd40fa07e03c7e1
      Gerrit-Change-Number: 9210
      Gerrit-PatchSet: 2
      Gerrit-Owner: Damien Martin-guillerez <dmar...@google.com>
      Gerrit-Reviewer: Damien Martin-guillerez <dmar...@google.com>
      Gerrit-Reviewer: Lukács T. Berki <lbe...@google.com>
      Gerrit-Reviewer: Vladimir Moskva <vla...@google.com>

      Vladimir Moskva (Gerrit)

      unread,
      Mar 6, 2017, 9:45:58 AM3/6/17
      to Damien Martin-guillerez, Lukács T. Berki

      Vladimir Moskva uploaded patch set #2 to the change originally created by Damien Martin-guillerez.

      View Change

      Flag to import external repositories in python import path 
      
      This flag will be used to turn off the feature until we
      get support for --incompatible flag.
      
      This flag is going to go away very fast, do not rely on it too much.
      
      To be cherry-picked for 0.4.5 (#2472)
      
      --
      Change-Id: I2d3c79ae0c2c53089677573cffd40fa07e03c7e1
      Reviewed-on: https://cr.bazel.build/9210
      PiperOrigin-RevId: 149291628
      MOS_MIGRATED_REVID=149291628
      ---
      M src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonConfiguration.java
      M src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPythonSemantics.java
      M src/main/java/com/google/devtools/build/lib/bazel/rules/python/stub_template.txt
      3 files changed, 24 insertions(+), 6 deletions(-)
      
      

      To view, visit change 9210. To unsubscribe, visit settings.

      Gerrit-Project: bazel
      Gerrit-Branch: master
      Gerrit-MessageType: newpatchset
      Gerrit-Change-Id: I2d3c79ae0c2c53089677573cffd40fa07e03c7e1
      Gerrit-Change-Number: 9210
      Reply all
      Reply to author
      Forward
      0 new messages