Rake does not recognize rules with multiple extensions - rake

I generate PDFs from Markdown files using Rake. If a Markdown file is filename.md, I like the PDF to be filename.md.pdf not filename.pdf, so that autocompletion works the way I like and so that it's clear what the source of the PDF file is.
I have this Rake file, which works fine.
MDFILES = FileList["*.md"]
PDFS = MDFILES.ext("pdf")
desc "Build PDFs of all chapters"
task :pdfs => PDFS
# Build PDFs from Markdown source
rule ".pdf" => ".md" do |t|
sh "pandoc #{t.source} -o #{t.name}"
end
If I run rake pdfs or rake filename.pdf the PDFs are generated as expected, but the PDFs are named filename.pdf.
But I want the Rakefile to be this instead:
MDFILES = FileList["*.md"]
PDFS = MDFILES.ext("md.pdf")
desc "Build PDFs of all chapters"
task :pdfs => PDFS
# Build PDFs from Markdown source
rule "md.pdf" => ".md" do |t|
sh "pandoc #{t.source} -o #{t.name}"
end
Running rake pdfs or rake filename.md.pdf returns the error Don't know how to build task 'filename.md.pdf'.
How can I produce filenames the way I want?
By the way, this type of rule works fine with Make, to wit:
%.md.pdf : %.md
pandoc $< -o $#

I've had a similar problem myself recently when I attempted to specify an extension with multiple dots in a rule. I solved it by using a different rule syntax as described here.
Try something like this for your rule:
rule( /\.md\.pdf$/ => [
proc {|task_name| task_name.sub(/\.md\.pdf$/, '.md') }
]) do |t|
sh "pandoc #{t.source} -o #{t.name}"
end

Related

rake executes file task when file already exists

I have a rakefile that executes some (but not all) of it's file tasks even if the files of interest have already been built. The frustrating thing, is that paring down my rake file to a MWE resolves the problem---even though I haven't altered anything wrt the filetask definition, how the files are being selected, the dependencies, or anything else. It seems that simply removing other (file)tasks from the rakefile remedies the problem.
I realize this is a really awful question, but does anyone have ideas about what might be going on here? I'd post sample code, but my MWE works as expected and I don't have any sense for what is causing the problem in the full rake file. All I can think to do is demonstrate that my MWE is literally an excerpt from the full Rakefile, unaltered...
➜ solutionmaps cat mwe/Rakefile|sed '/^$/d'|tee a
require 'rake'
require 'rake/clean'
require 'pathname'
HOME = ENV['HOME']
SHARED_ATLAS = "#{HOME}/MRI/Manchester/data/CommonBrains/MNI_EPI_funcRes.nii"
TXT = Rake::FileList["txt/nodestrength/??.mni"]
AFNI_RAW = TXT.pathmap("afni/nodestrength/%n_raw+tlrc.HEAD")
AFNI_RAW.zip(TXT).each do |target,source|
file target => [source] do
sh("3dUndump -master #{SHARED_ATLAS} -xyz -datum float -prefix #{target.sub("+tlrc.HEAD","")} #{source}")
end
CLOBBER.push(target)
CLOBBER.push(target.sub(".HEAD",".BRIK"))
CLOBBER.push(target.sub(".HEAD",".BRIK.gz"))
end
➜ solutionmaps perl -ne 'print if ($seen{$_} .= #ARGV) =~ /10$/' Rakefile mwe/Rakefile|sed '/^$/d'|tee b
require 'rake'
require 'rake/clean'
require 'pathname'
HOME = ENV['HOME']
SHARED_ATLAS = "#{HOME}/MRI/Manchester/data/CommonBrains/MNI_EPI_funcRes.nii"
TXT = Rake::FileList["txt/nodestrength/??.mni"]
AFNI_RAW = TXT.pathmap("afni/nodestrength/%n_raw+tlrc.HEAD")
AFNI_RAW.zip(TXT).each do |target,source|
file target => [source] do
sh("3dUndump -master #{SHARED_ATLAS} -xyz -datum float -prefix #{target.sub("+tlrc.HEAD","")} #{source}")
end
CLOBBER.push(target)
CLOBBER.push(target.sub(".HEAD",".BRIK"))
CLOBBER.push(target.sub(".HEAD",".BRIK.gz"))
end
➜ solutionmaps diff a b
➜ solutionmaps
And that my mwe works as expected (that is, it does not execute the file task).
➜ mwe rake --trace --dry-run afni/nodestrength/02_raw+tlrc.HEAD
** Invoke afni/nodestrength/02_raw+tlrc.HEAD (first_time, not_needed)
** Invoke txt/nodestrength/02.mni (first_time, not_needed)
But the full rakefile does not.
rake --trace --dry-run afni/nodestrength/02_raw+tlrc.HEAD
** Invoke afni/nodestrength/02_raw+tlrc.HEAD (first_time)
** Invoke txt/nodestrength/02.mni (first_time, not_needed)
** Execute (dry run) afni/nodestrength/02_raw+tlrc.HEAD
➜ solutionmaps ls afni/nodestrength/02_raw+tlrc.HEAD
afni/nodestrength/02_raw+tlrc.HEAD
Finally happened across a possible answer:
Rake determines that a file task needs to be run if the file doesn’t exist or if any of the prerequisite file tasks are newer.
Quoted from: http://madewithenvy.com/ecosystem/articles/2013/rake-file-tasks/
Since my Rakefiles are under heavy development, and my Filetasks are all pretty interrelated, this is probably why my Rakefile always wanted to rebuild everything.

Is there a way to tell django compressor to create source maps

I want to be able to debug minified compressed javascript code on my production site. Our site uses django compressor to create minified and compressed js files. I read recently about chrome being able to use source maps to help debug such javascript. However I don't know how/if possible to tell the django compressor to create source maps when compressing the js files
I don't have a good answer regarding outputting separate source map files, however I was able to get inline working.
Prior to adding source maps my settings.py file used the following precompilers
COMPRESS_PRECOMPILERS = (
('text/coffeescript', 'coffee --compile --stdio'),
('text/less', 'lessc {infile} {outfile}'),
('text/x-sass', 'sass {infile} {outfile}'),
('text/x-scss', 'sass --scss {infile} {outfile}'),
('text/stylus', 'stylus < {infile} > {outfile}'),
)
After a quick
$ lessc --help
You find out you can put the less and map files in to the output css file. So my new text/less precompiler entry looks like
('text/less', 'lessc --source-map-less-inline --source-map-map-inline {infile} {outfile}'),
Hope this helps.
Edit: Forgot to add, lessc >= 1.5.0 required for this, to upgrade use
$ [sudo] npm update -g less
While I couldn't get this to work with django-compressor (though it should be possible, I think I just had issues getting the app set up correctly), I was able to get it working with django-assets.
You'll need to add the appropriate command-line argument to the less filter source code as follows:
diff --git a/src/webassets/filter/less.py b/src/webassets/filter/less.py
index eb40658..a75f191 100644
--- a/src/webassets/filter/less.py
+++ b/src/webassets/filter/less.py
## -80,4 +80,4 ## class Less(ExternalTool):
def input(self, in_, out, source_path, **kw):
# Set working directory to the source file so that includes are found
with working_directory(filename=source_path):
- self.subprocess([self.less or 'lessc', '-'], out, in_)
+ self.subprocess([self.less or 'lessc', '--line-numbers=mediaquery', '-'], out, in_)
Aside from that tiny addition:
make sure you've got the node -- not the ruby gem -- less compiler (>=1.3.2 IIRC) available in your path.
turn on the sass source-maps option buried away in chrome's web inspector config pages. (yes, 'sass' not less: less tweaked their debug-info format to match sass's since since sass had already implemented a chrome-compatible mapping and their formats weren't that different to begin with anyway...)
Not out of the box but you can extend a custom filter:
from compressor.filters import CompilerFilter
class UglifyJSFilter(CompilerFilter):
command = "uglifyjs -c -m " /
"--source-map-root={relroot}/ " /
"--source-map-url={name}.map.js" /
"--source-map={relpath}/{name}.map.js -o {output}"

Rake -- Watch for Changes

Is there a way using Rake to watch depedencies for changes and execute tasks automatically?
For example, given the following Rakefile:
file 'main.o' => ["main.c", "greet.h"] do
sh "cc -c -o main.o main.c"
end
file 'greet.o' => ['greet.c'] do
sh "cc -c -o greet.o greet.c"
end
file "hello" => ["main.o", "greet.o"] do
sh "cc -o hello main.o greet.o"
end
It would be great if when I changed greet.c or main.c then hello would automatically execute.
AFAIK rake doesn't support watching files and directories with a continuous compilation approach.
The buildr project implements a continuous compilation task for java which you can get inspired by to do the job for your project.
Also, it looks like guard might help there. See the guard github profile for implementations using guard. There's even a Rake extension to do what you want!
There is also rerun https://github.com/alexch/rerun. You can do rerun rake to watch and run the default task.

How to change the order of rsync in symfony deployment task

I want to deploy a part of my symfony application, say, it's like a module.
I want to exclude all files first, and then include only the files of
my new module.
For deployment I use the following symfony task
php symfony project:deploy production -t
The parameter -t prints all files to the output that are included in this dry run of rsync.
Content of config/rsync_exclude.txt is only *, since I like to exclude everthing:
*
In config/rsync_include.txt I list all the files and folders for the inclusion:
config/
config/mysupermodule.yml
lib/model/doctrine/
lib/model/doctrine/MySuperclass.php
lib/model/doctrine/MySuperclassTable.php
lib/
lib/MySuperLibrary/
lib/MySuperLibrary/*
The symfony task builds the following rsync command:
rsync --dry-run -azC --force --delete --progress --exclude-from=config/rsync_exclude.txt --include-from=config/rsync_include.txt -e "ssh -p22" ./ user#www.server.com:/test_deployment/
Problem 1: The the task doesn't sync any files.
Solution to 1: Change order: Include first, then exclude.
I figured out, that if I change my need to this one:
I want to include all files of my new module and exclude then all
other.
This means using the following command:
rsync --dry-run -azC --force --delete --progress --include-from=config/rsync_include.txt --exclude-from=config/rsync_exclude.txt -e "ssh -p22" ./ user#www.server.com:/test_deployment/
The rsync works.
Problem 2: How can I change the order of the rsync when using the symfony task?
The symfony task first excludes than includes.
Solution 2: ?
It is NOT possible.
But you can edit the deployment task in lib/task/project/sfProjectDeployTask.class.php.
Replace this (line 145 to 154 in SF 1.4):
if (file_exists($options['rsync-dir'].'/rsync_exclude.txt'))
{
$parameters .= sprintf(' --exclude-from=%s/rsync_exclude.txt', $options['rsync-dir']);
}
if (file_exists($options['rsync-dir'].'/rsync_include.txt'))
{
$parameters .= sprintf(' --include-from=%s/rsync_include.txt', $options['rsync-dir']);
}
with this:
if (file_exists($options['rsync-dir'].'/rsync_include.txt'))
{
$parameters .= sprintf(' --include-from=%s/rsync_include.txt', $options['rsync-dir']);
}
if (file_exists($options['rsync-dir'].'/rsync_exclude.txt'))
{
$parameters .= sprintf(' --exclude-from=%s/rsync_exclude.txt', $options['rsync-dir']);
}
In short: switch this two IF statements.
Let's change the way you want to do.
You should use only the exclude file. Exclude only directories that changed but you don't want to sync.
Because anyway if you modules/, app/, ... directories haven't change, you don't have to put them in the exclude file because they will remain the same on both server.

How do I loop over several files, keeping the base name for further processing?

I have multiple text files that need to be tokenised, POS and NER. I am using C&C taggers and have run their tutorial, but I am wondering if there is a way to tag multiple files rather than one by one.
At the moment I am tokenising the files:
bin/tokkie --input working/tutorial/example.txt--quotes delete --output working/tutorial/example.tok
as follows and then Part of Speech tagging:
bin/pos --input working/tutorial/example.tok --model models/pos --output working/tutorial/example.pos
and lastly Named Entity Recognition:
bin/ner --input working/tutorial/example.pos --model models/ner --output working/tutorial/example.ner
I am not sure how I would go about creating a loop to do this and keep the file name the same as the input but with the extension representing the tagging it has. I was thinking of a bash script or perhaps Perl to open the directory but I am not sure on how to enter the C&C commands in order for the script to understand.
At the moment I am doing it manually and it's pretty time consuming to say the least!
Untested, likely needs some directory mangling.
use autodie qw(:all);
use File::Basename qw(basename);
for my $text_file (glob 'working/tutorial/*.txt') {
my $base_name = basename($text_file, '.txt');
system 'bin/tokkie',
'--input' => "working/tutorial/$base_name.txt",
'--quotes' => 'delete',
'--output' => "working/tutorial/$base_name.tok";
system 'bin/pos',
'--input' => "working/tutorial/$base_name.tok",
'--model' => 'models/pos',
'--output' => "working/tutorial/$base_name.pos";
system 'bin/ner',
'--input' => "working/tutorial/$base_name.pos",
'--model' => 'models/ner',
'--output' => "working/tutorial/$base_name.ner";
}
In Bash:
#!/bin/bash
dir='working/tutorial'
for file in "$dir"/*.txt
do
noext=${file/%.txt}
bin/tokkie --input "$file" --quotes delete --output "$noext.tok"
bin/pos --input "$noext.tok" --model models/pos --output "$noext.pos"
bin/ner --input "$noext.pos" --model models/ner --output "$noext.ner"
done