I want to create a temporary folder with a directory and some files:
import os
import tempfile
from pathlib import Path
with tempfile.TemporaryDirectory() as tmp_dir:
# generate some random files in it
Path('file.txt').touch()
Path('file2.txt').touch()
files_in_dir = os.listdir(tmp_dir)
print(files_in_dir)
Expected: [file.txt,file2.txt]
Result: []
Does anyone know how to this in Python? Or is there a better way to just do some mocking?
You have to create the file inside the directory by getting the path of your tmp_dir. The with context does not do that for you.
with tempfile.TemporaryDirectory() as tmp_dir:
Path(tmp_dir, 'file.txt').touch()
Path(tmp_dir, 'file2.txt').touch()
files_in_dir = os.listdir(tmp_dir)
print(files_in_dir)
# ['file2.txt', 'file.txt']
Related
Does ruamel.yaml have a function to do the process with all files in one directory?
Something like this:
data = yaml.load(Path("*.*"))
No, it does not, but you can do it one line (assuming you have the imports and the YAML() instance):
from pathlib import Path
import ruamel.yaml
yaml = ruamel.yaml.YAML()
data = [yaml.load(p) for p in Path('.').glob('*.yaml')]
I read a mf4 file using asammdf package. I want to delete this file after doing some edits but failed. The error shows: This file is in use by another program and cannot be accessed by the process. I wonder how to close a mf4 file before delete.
from asammdf import MDF
mdf = MDF(path)
from asammdf import MDF
mdf = MDF(path)
# .... something something
mdf.close()
The project lib documentation shows how to save a pandas dataframe to the project assets:
# Import the lib
from project_lib import Project
project = Project(sc,"<ProjectId>", "<ProjectToken>")
# let's assume you have the pandas DataFrame pandas_df which contains the data
# you want to save in your object storage as a csv file
project.save_data("file_name.csv", pandas_df.to_csv())
# the function returns a dict which contains the asset_id, bucket_name and file_name
# upon successful saving of the data
However, if I have a local file ...
! wget url_to_binary_file
How can I then upload that file to the project’s assets?
I needed to read the file as bytes. Note that this will read the file into memory, don’t try this is you have a file that is larger than your available memory:
import io
filename = ‘thefilename’
with open(filename, 'rb') as z:
data = io.BytesIO(z.read())
project.save_data(
filename, data, set_project_asset=True, overwrite=True
)
I want to simplify code. so i make a utils.py , but Google Colaboratory directory is "/content" I read other questions. but this is not my solution
In Google's Colab notebook, How do I call a function from a Python file?
%%writefile example.py
def f():
print 'This is a function defined in a Python source file.'
# Bring the file into the local Python environment.
execfile('example.py')
f()
This is a function defined in a Python source file.
It look likes just using def().
using this, i always write the code in cell.
but i want to this code
import example.py
example.f()
A sample maybe you want:
!wget https://raw.githubusercontent.com/tensorflow/models/master/samples/core/get_started/iris_data.py -P local_modules -nc
import sys
sys.path.append('local_modules')
import iris_data
iris_data.load_data()
I have also had this problem recently.
I addressed the issue by the following steps, though it's not a perfect solution.
src = list(files.upload().values())[0]
open('util.py','wb').write(src)
import util
This code should work with Python 3:
from google.colab import drive
import importlib.util
# Mount your drive. It will be at this path: "/content/gdrive/My Drive/"
drive.mount('/content/gdrive')
# Load your module
spec = importlib.util.spec_from_file_location("YOUR_MODULE_NAME", "/content/gdrive/My Drive/utils.py")
your_module_name = importlib.util.module_from_spec(spec)
spec.loader.exec_module(your_module_name)
import importlib.util
import sys
from google.colab import drive
drive.mount('/content/gdrive')
# To add a directory with your code into a list of directories
# which will be searched for packages
sys.path.append('/content/gdrive/My Drive/Colab Notebooks')
import example.py
This works for me.
Use this if you are out of content folder! hope this help!
import sys
sys.path.insert(0,'/content/my_project')
from example import*
STEP 1. I have just created a folder 'common_module' like shown in the image :
STEP 2 called the required Class from my "colab" code cell,
sys.path.append('/content/common_module/')
from DataPreProcessHelper import DataPreProcessHelper as DPPHelper
My class file 'DataPreProcessHelper.py' looks like this
Add path of 'sample.py' file to system paths as:
import sys
sys.path.append('drive/codes/')
import sample
While trying to use the sphinx matlab domain I can't get the MWE to work, provided on the extensions pypi site
There is always this Can't import module error. I'd guess, that the extension kind of generates pseudo modules from the m-code, but up to know I actually could not figure out, how this mechanism works.
The dir structure looks like this
root
|--test_data
| |--MyHandleClass.m
|
|--doc
|--------conf.py
|--------Makefile
|--------index.rst
The files MyHandleClass.m and index.rst contain the example code given on the package site and the conf.py starts like this
import sys, os
sys.path.append(os.path.abspath('.'))
sys.path.append(os.path.abspath('./test_data'))
# -- General configuration -----------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
"sphinxcontrib.matlab",
"sphinx.ext.autosummary",
"sphinx.ext.autodoc"]
autodoc_default_flags = ['members','show-inheritance','undoc-members']
autoclass_content = 'both'
mathjax_path = 'http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=default'
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8'
# The master toctree document.
master_doc = 'index'
Error msg
WARNING: autodoc: failed to import module u'test_data'; the following exception was raised:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\sphinx\ext\autodoc.py", line 335, in import_object
__import__(self.modname)
ImportError: No module named test_data
E:\ME\doc\index.rst:13: WARNING: don't know which module to import for autodocumenting u'MyHandleClass' (try placing a "module" or "currentmodule" directive in the document, or giving an explicit module name)
After varying this and that maybe somebody out there has a clue?
Thanks for trying the matlabdomain sphinxcontrib extension. In order to use Sphinx to document MATLAB m-files, you need to add matlab_src_dir in conf.py as described in the Configuration section of the documenation. This is because the Python interpreter can't import a MATLAB m-file. Therefore you should not add your MATLAB root to the Python sys.path, or you will get the error you received. Instead set matlab_src_dir to the path containing the folder of your MATLAB project which you want to document.
Given your file structure, in order to document test_data use a conf.py with the following:
import os
# NOTE: don't add MATLAB m-files to `sys.path`
#sys.path.insert(0, os.path.abspath('.'))
# instead add them to `matlab_src_dir
matlab_src_dir = os.path.abspath('..') # MATLAB
Hope that does it! Please feel free to ask any more questions. I'm happy to help!