Related
I tried creating a Diagram using the NarcolepticSuperhero machine defined in the GitHub documentation but it only outputs this:
Steps to recreate:
Create a file named test.py, with this content:
from transitions import Machine
from transitions.extensions import GraphMachine
import random
class NarcolepticSuperhero(object):
# Define some states. Most of the time, narcoleptic superheroes are just like
# everyone else. Except for...
states = ['asleep', 'hanging out', 'hungry', 'sweaty', 'saving the world']
def __init__(self, name):
# No anonymous superheroes on my watch! Every narcoleptic superhero gets
# a name. Any name at all. SleepyMan. SlumberGirl. You get the idea.
self.name = name
# What have we accomplished today?
self.kittens_rescued = 0
# Initialize the state machine
self.machine = Machine(model=self, states=NarcolepticSuperhero.states, initial='asleep')
# Add some transitions. We could also define these using a static list of
# dictionaries, as we did with states above, and then pass the list to
# the Machine initializer as the transitions= argument.
# At some point, every superhero must rise and shine.
self.machine.add_transition(trigger='wake_up', source='asleep', dest='hanging out')
# Superheroes need to keep in shape.
self.machine.add_transition('work_out', 'hanging out', 'hungry')
# Those calories won't replenish themselves!
self.machine.add_transition('eat', 'hungry', 'hanging out')
# Superheroes are always on call. ALWAYS. But they're not always
# dressed in work-appropriate clothing.
self.machine.add_transition('distress_call', '*', 'saving the world',
before='change_into_super_secret_costume')
# When they get off work, they're all sweaty and disgusting. But before
# they do anything else, they have to meticulously log their latest
# escapades. Because the legal department says so.
self.machine.add_transition('complete_mission', 'saving the world', 'sweaty',
after='update_journal')
# Sweat is a disorder that can be remedied with water.
# Unless you've had a particularly long day, in which case... bed time!
self.machine.add_transition('clean_up', 'sweaty', 'asleep', conditions=['is_exhausted'])
self.machine.add_transition('clean_up', 'sweaty', 'hanging out')
# Our NarcolepticSuperhero can fall asleep at pretty much any time.
self.machine.add_transition('nap', '*', 'asleep')
def update_journal(self):
""" Dear Diary, today I saved Mr. Whiskers. Again. """
self.kittens_rescued += 1
#property
def is_exhausted(self):
""" Basically a coin toss. """
return random.random() < 0.5
def change_into_super_secret_costume(self):
print("Beauty, eh?")
batman = NarcolepticSuperhero("Batman")
batman.wake_up()
batman.state
machine = GraphMachine(model=batman)
batman.get_graph().draw("test.png", prog='dot')
Install the requirements (Ubuntu 20.10, I tested on it's docker image) and run the script:
$ apt install graphviz graphviz-dev
$ pip3 install transitions graphviz pygraphviz
$ python3 test.py
Check the generated image
You are instantiating a GraphMachine without any states and transitions here:
machine = GraphMachine(model=batman)
You basically reuse NarcolepticSuperhero as a model for that new machine but what you should do instead is changing the NarcolepticSuperhero's machine into a GraphMachine:
# Initialize the state machine
self.machine = GraphMachine(model=self, states=NarcolepticSuperhero.states, initial='asleep')
# [...]
batman.state
# this is not required anymore
# machine = GraphMachine(model=batman)
batman.get_graph().draw("test.png", prog='dot')
I am trying to add a procedure to pop-up a modal dialog inside a plug-in.
Its purpose is to query a response at designated steps within the control-flow of the plug-in (not just acquire parameters at its start).
I have tried using gtk - I get a dialog but it is asynchronous - the plugin continues execution. It needs to operate as a synchronous function.
I have tried registering a plugin in order to take advantage of the gimpfu start-up dialogue for same. By itself, it works; it shows up in the procedural db when queried. But I never seem to be able to actually invoke it from within another plug-in - its either an execution error or wrong number of arguments no matter how many permutations I try.
[Reason behind all of this nonsense: I have written a lot of extension Python scripts for PaintShopPro. I have written a App package (with App.Do, App.Constants, Environment and the like that lets me begin to port those scripts to GIMP -- yes it is perverse, and yes sometimes the code just has to be rewritten, but for a lot of what I actual use in the PSP.API it is sufficient.
However, debugging and writing the module rhymes with witch. So. I am trying to add emulation of psp's "SetExecutionMode" (ie interactive). If
set, the intended behavior is that the App.Do() method will "pause" after/before it runs the applicable psp emulation code by popping up a simple message dialog.]
A simple modal dialogue within a gimp python-fu plug-in can be implemented via gtk's Dialog interface, specifically gtk.MessageDialog.
A generic dialog can be created via
queryDialogue = gtk.MessageDialog(None, gtk.DIALOG_DESTROY_WITH_PARENT \
gtk.MESSAGE_QUESTION, \
gtk.BUTTONS_OK_CANCEL, "")
Once the dialog has been shown,
a synchronous response may be obtained from it
queryDialogue.show()
response = queryDialogue.run()
queryDialogue.hide()
The above assumes that the dialog is not created and thence destroyed after each use.
In the use case (mentioned in the question) of a modal dialog to manage single stepping through a pspScript in gimp via an App emulator package, the dialogue message contents need to be customized for each use. [Hence, the "" for the message argument in the Constructor. [more below]]
In addition, the emulator must be able to accept a [cancel] response to 'get out of Dodge' - ie quit the entire plug-in (gracefully). I could not find a gimpfu interface for the latter, (and do not want to kill the app entirely via gimp.exit()). Hence, this is accomplished by raising a custom Exception class [appTerminate] within the App pkg and catching the exception in the outer-most scope of the plugin. When caught, then, the plug-in returns (exits).[App.Do() can not return a value to indicate continue/exit/etc, because the pspScripts are to be included verbatim.]
The following is an abbreviated skeleton of the solution -
a plug-in incorporating (in part) a pspScript
the App.py pkg supplying the environment and App.Do() to support the pspScript
a Map.py pkg supporting how pspScripts use dot-notation for parameters
App.py demonstrates creation, customization and use of a modal dialog - App.doContinue() displays the dialogue illustrating how it can be customized on each use.
App._parse() parses the pspScript (excerpt showing how it determines to start/stop single-step via the dialogue)
App._exec() implements the pspScript commands (excerpt showing how it creates the dialogue, identifies the message widget for later customization, and starts/stops its use)
# App.py (abbreviated)
#
import gimp
import gtk
import Map # see https://stackoverflow.com/questions/2352181/how-to- use-a-dot-to-access-members-of-dictionary
from Map import *
pdb = gimp.pdb
isDialogueAvailable = False
queryDialogue = None
queryMessage = None
Environment = Map({'executionMode' : 1 })
_AutoActionMode = Map({'Match' : 0})
_ExecutionMode = Map({'Default' : 0}, Silent=1, Interactive=2)
Constants = Map({'AutoActionMode' : _AutoActionMode}, ExecutionMode=_ExecutionMode ) # etc...
class appTerminate(Exception): pass
def Do(eNvironment, procedureName, options = {}):
global appTerminate
img = gimp.image_list()[0]
lyr = pdb.gimp_image_get_active_layer(img)
parsed = _parse(img, lyr, procedureName, options)
if eNvironment.executionMode == Constants.ExecutionMode.Interactive:
resp = doContinue(procedureName, parsed.detail)
if resp == -5: # OK
print procedureName # log to stdout
if parsed.valid:
if parsed.isvalid:
_exec(img, lyr, procedureName, options, parsed, eNvironment)
else:
print "invalid args"
else:
print "invalid procedure"
elif resp == -6: # CANCEL
raise appTerminate, "script cancelled"
pass # terminate plugin
else:
print procedureName + " skipped"
pass # skip execution, continue
else:
_exec(img, lyr, procedureName, options, parsed, eNvironment)
return
def doContinue(procedureName, details):
global queryMessage, querySkip, queryDialogue
# - customize the dialog -
if details == "":
msg = "About to execute procedure \n "+procedureName+ "\n\nContinue?"
else:
msg = "About to execute procedure \n "+procedureName+ "\n\nDetails - \n" + details +"\n\nContinue?"
queryMessage.set_text(msg)
queryDialogue.show()
resp = queryDialogue.run() # get modal response
queryDialogue.hide()
return resp
def _parse(img, lyr, procedureName, options):
# validate and interpret App.Do options' semantics vz gimp
if procedureName == "Selection":
isValid=True
# ...
# parsed = Map({'valid' : True}, isvalid=True, start=Start, width=Width, height=Height, channelOP=ChannelOP ...
# /Selection
# ...
elif procedureName == "SetExecutionMode":
generalOptions = options['GeneralSettings']
newMode = generalOptions['ExecutionMode']
if newMode == Constants.ExecutionMode.Interactive:
msg = "set mode interactive/single-step"
else:
msg = "set mode silent/run"
parsed = Map({'valid' : True}, isvalid=True, detail=msg, mode=newMode)
# /SetExecutionMode
else:
parsed = Map({'valid' : False})
return parsed
def _exec(img, lyr, procedureName, options, o, eNvironment):
global isDialogueAvailable, queryMessage, queryDialogue
#
try:
# -------------------------------------------------------------------------------------------------------------------
if procedureName == "Selection":
# pdb.gimp_rect_select(img, o.start[0], o.start[1], o.width, o.height, o.channelOP, ...
# /Selection
# ...
elif procedureName == "SetExecutionMode":
generalOptions = options['GeneralSettings']
eNvironment.executionMode = generalOptions['ExecutionMode']
if eNvironment.executionMode == Constants.ExecutionMode.Interactive:
if isDialogueAvailable:
queryDialogue.destroy() # then clean-up and refresh
isDialogueAvailable = True
queryDialogue = gtk.MessageDialog(None, gtk.DIALOG_DESTROY_WITH_PARENT, gtk.MESSAGE_QUESTION, gtk.BUTTONS_OK_CANCEL, "")
queryDialogue.set_title("psp/APP.Do Emulator")
queryDialogue.set_size_request(450, 180)
aqdContent = queryDialogue.children()[0]
aqdHeader = aqdContent.children()[0]
aqdMsgBox = aqdHeader.children()[1]
aqdMessage = aqdMsgBox.children()[0]
queryMessage = aqdMessage
else:
if isDialogueAvailable:
queryDialogue.destroy()
isDialogueAvailable = False
# /SetExecutionMode
else: # should not get here (should have been screened by parse)
raise AssertionError, "unimplemented PSP procedure: " + procedureName
except:
raise AssertionError, "App.Do("+procedureName+") generated an exception:\n" + sys.exc_info()
return
A skeleton of the plug-in itself. This illustrates incorporating a pspScript which includes a request for single-step/interactive execution mode, and thus the dialogues. It catches the terminate exception raised via the dialogue, and then terminates.
def generateWebImageSet(dasImage, dasLayer, title, mode):
try:
img = dasImage.duplicate()
# ...
bkg = img.layers[-1]
frameWidth = 52
start = bkg.offsets
end = (start[0]+bkg.width, start[1]+frameWidth)
# pspScript: (snippet included verbatim)
# SetExecutionMode / begin interactive single-step through pspScript
App.Do( Environment, 'SetExecutionMode', {
'GeneralSettings': {
'ExecutionMode': App.Constants.ExecutionMode.Interactive
}
})
# Selection
App.Do( Environment, 'Selection', {
'General' : {
'Mode' : 'Replace',
'Antialias' : False,
'Feather' : 0
},
'Start': start,
'End': end
})
# Promote
App.Do( Environment, 'SelectPromote' )
# und_so_weiter ...
except App.appTerminate:
raise AssertionError, "script cancelled"
# /generateWebImageSet
# _generateFloatingCanvasSetWeb.register -----------------------------------------
#
def generateFloatingCanvasSetWeb(dasImage, dasLayer, title):
mode="FCSW"
generateWebImageSet(dasImage, dasLayer, title, mode)
register(
"generateFloatingCanvasSetWeb",
"Generate Floating- Frame GW Canvas Image Set for Web Page",
"Generate Floating- Frame GW Canvas Image Set for Web Page",
"C G",
"C G",
"2019",
"<Image>/Image/Generate Web Imagesets/Floating-Frame Gallery-Wrapped Canvas Imageset...",
"*",
[
( PF_STRING, "title", "title", "")
],
[],
generateFloatingCanvasSetWeb)
main()
I realize that this may seem like a lot of work just to be able to include some pspScripts in a gimp plug-in, and to be able to single-step through the emulation. But we are talking about maybe 10K lines of scripts (and multiple scripts).
However, if any of this helps anyone else with dialogues inside plug-ins, etc., so much the better.
Are there minimal, or even larger, working examples of using SCons and knitr to generate reports from .Rmd files?
kniting an cleaning_session.Rmd file from the command line (bash shell) to derive an .html file, may be done via:
Rscript -e "library(knitr); knit('cleaning_session.Rmd')".
In this example, Rscript and instructions are fed to a Makefile:
RMDFILE=test
html :
Rscript -e "require(knitr); require(markdown); knit('$(RMDFILE).rmd', '$(RMDFILE).md'); markdownToHTML('$(RMDFILE).md', '$(RMDFILE).html', options=c('use_xhtml', 'base64_images')); browseURL(paste('file://', file.path(getwd(),'$(RMDFILE).html'), sep=''
In this answer https://stackoverflow.com/a/10945832/1172302, there is reportedly a solution using SCons. Yet, I did not test enough to make it work for me. Essentially, it would be awesome to have something like the example presented at https://tex.stackexchange.com/a/26573/8272.
[Updated] One working example is an Sconstruct file:
import os
environment = Environment(ENV=os.environ)
# define a `knitr` builder
builder = Builder(action = '/usr/local/bin/knit $SOURCE -o $TARGET',
src_suffix='Rmd')
# add builders as "Knit", "RMD"
environment.Append( BUILDERS = {'Knit' : builder} )
# define an `rmarkdown::render()` builder
builder = Builder(action = '/usr/bin/Rscript -e "rmarkdown::render(input=\'$SOURCE\', output_file=\'$TARGET\')"',
src_suffix='Rmd')
environment.Append( BUILDERS = {'RMD' : builder} )
# define source (and target files -- currently useless, since not defined above!)
# main cleaning session code
environment.RMD(source='cleaning_session.Rmd', target='cleaning_session.html')
# documentation of the Cleaning Process
environment.Knit(source='Cleaning_Process.Rmd', target='Cleaning_Process.html')
# documentation of data
environment.Knit(source='Code_Book.Rmd', target='Code_Book.html')
The first builder calls the custom script called knit. Which, in turn, takes care of the target file/extension, here being cleaning_session.html. Likely the suffix parameter is not needed altogether, in this very example.
The second builder added is Rscript -e "rmarkdown::render(\'$SOURCE\')"'.
The existence of $TARGETs (as in the example at Command wrapper) ensures SCons won't repeat work if a target file already exists.
The custom script (whose source I can't retrieve currently) is:
#!/usr/bin/env Rscript
local({
p = commandArgs(TRUE)
if (length(p) == 0L || any(c('-h', '--help') %in% p)) {
message('usage: knit input [input2 input3] [-n] [-o output output2 output3]
-h, --help to print help messages
-n, --no-convert do not convert tex to pdf, markdown to html, etc
-o output filename(s) for knit()')
q('no')
}
library(knitr)
o = match('-o', p)
if (is.na(o)) output = NA else {
output = tail(p, length(p) - o)
p = head(p, o - 1L)
}
nc = c('-n', '--no-convert')
knit_fun = if (any(nc %in% p)) {
p = setdiff(p, nc)
knit
} else {
if (length(p) == 0L) stop('no input file provided')
if (grepl('\\.(R|S)(nw|tex)$', p[1])) {
function(x, ...) knit2pdf(x, ..., clean = TRUE)
} else {
if (grepl('\\.R(md|markdown)$', p[1])) knit2html else knit
}
}
mapply(knit_fun, p, output = output, MoreArgs = list(envir = globalenv()))
})
The only thing, now, necessary is to run scons.
Ive just moved my projects code from java.net to BitBucket. But my jira issue tracking is still hosted on java.net, although BitBucket does have some options for linking to an external issue tracker I don't think I can use it for java.net, not least because I do not have the admin priviledges need to install the DVCS connector.
So I thought an alternative option would be to export and then import the issues into BitBucket issue tracker, is that possible ?
Progress so far
So I tried following the steps in both informative answers using OSX below but I hit a problem - I'm rather confused about what the script would actually be called because in the answers it talks about export.py but no such script exists with that name so I renamed the one I downloaded.
sudo easy_install pip (OSX)
pip install jira
pip install configparser
easy_install -U setuptools
Go to https://bitbucket.org/reece/rcore, select downloads tab, download zip and unzip, and rename to reece ( for some reason git clone https://bitbucket.org/reece/rcore fails with error)
cd reece/rcore
Save script as export.py in rcore subfolder
Replace iteritems with items in import.py
Replace iteritems with types/immutabledict.py
Create .config in rcore folder
Create .config/jira-issues-move-to-bitbucket.conf containing
jira-username=paultaylor
jira-hostname=https://java.net/jira/browse/JAUDIOTAGGER
jira-password=password
Run python export.py --jira-project jaudiotagger
gives
macbook:rcore paul$ python export.py --jira-project jaudiotagger
Traceback (most recent call last):
File "export.py", line 24, in <module>
import configparser
ImportError: No module named configparser
- Run python export.py --jira-project jaudiotagger
I need to run pip insdtall as root so did
sudo pip install configparser
and that worked
but now
python export.py --jira.project jaudiotagger
gives
File "export.py" line 35, in <module?
from jira.client import JIRA
ImportError: No module named jira.client
You can import issues into BitBucket, they just need to be in the appropriate format. Fortunately, Reece Hart has already written a Python script to connect to a Jira instance and export the issues.
To get the script to run I had to install the Jira Python package as well as the latest version of rcore (if you use pip you get an incompatible previous version, so you have to get the source). I also had to replace all instances of iteritems with items in the script and in rcore/types/immutabledict.py to make it work with Python 3. You will also need to fill in the dictionaries (priority_map, person_map, etc) with the values your project uses. Finally, you need a config file to exist with the connection info (see comments at the top of the script).
The basic command line usage is export.py --jira-project <project>
Once you've got the data exported, see the instructions for importing issues to BitBucket
#!/usr/bin/env python
"""extract issues from JIRA and export to a bitbucket archive
See:
https://confluence.atlassian.com/pages/viewpage.action?pageId=330796872
https://confluence.atlassian.com/display/BITBUCKET/Mark+up+comments
https://bitbucket.org/tutorials/markdowndemo/overview
2014-04-12 08:26 Reece Hart <reecehart#gmail.com>
Requires a file ~/.config/jira-issues-move-to-bitbucket.conf
with content like
[default]
jira-username=some.user
jira-hostname=somewhere.jira.com
jira-password=ur$pass
"""
import argparse
import collections
import configparser
import glob
import itertools
import json
import logging
import os
import pprint
import re
import sys
import zipfile
from jira.client import JIRA
from rcore.types.immutabledict import ImmutableDict
priority_map = {
'Critical (P1)': 'critical',
'Major (P2)': 'major',
'Minor (P3)': 'minor',
'Nice (P4)': 'trivial',
}
person_map = {
'reece.hart': 'reece',
# etc
}
issuetype_map = {
'Improvement': 'enhancement',
'New Feature': 'enhancement',
'Bug': 'bug',
'Technical task': 'task',
'Task': 'task',
}
status_map = {
'Closed': 'resolved',
'Duplicate': 'duplicate',
'In Progress': 'open',
'Open': 'new',
'Reopened': 'open',
'Resolved': 'resolved',
}
def parse_args(argv):
def sep_and_flatten(l):
# split comma-sep elements and flatten list
# e.g., ['a','b','c,d'] -> set('a','b','c','d')
return list( itertools.chain.from_iterable(e.split(',') for e in l) )
cf = configparser.ConfigParser()
cf.readfp(open(os.path.expanduser('~/.config/jira-issues-move-to-bitbucket.conf'),'r'))
ap = argparse.ArgumentParser(
description = __doc__
)
ap.add_argument(
'--jira-hostname', '-H',
default = cf.get('default','jira-hostname',fallback=None),
help = 'host name of Jira instances (used for url like https://hostname/, e.g., "instancename.jira.com")',
)
ap.add_argument(
'--jira-username', '-u',
default = cf.get('default','jira-username',fallback=None),
)
ap.add_argument(
'--jira-password', '-p',
default = cf.get('default','jira-password',fallback=None),
)
ap.add_argument(
'--jira-project', '-j',
required = True,
help = 'project key (e.g., JRA)',
)
ap.add_argument(
'--jira-issues', '-i',
action = 'append',
default = [],
help = 'issue id (e.g., JRA-9); multiple and comma-separated okay; default = all in project',
)
ap.add_argument(
'--jira-issues-file', '-I',
help = 'file containing issue ids (e.g., JRA-9)'
)
ap.add_argument(
'--jira-components', '-c',
action = 'append',
default = [],
help = 'components criterion; multiple and comma-separated okay; default = all in project',
)
ap.add_argument(
'--existing', '-e',
action = 'store_true',
default = False,
help = 'read existing archive (from export) and merge new issues'
)
opts = ap.parse_args(argv)
opts.jira_components = sep_and_flatten(opts.jira_components)
opts.jira_issues = sep_and_flatten(opts.jira_issues)
return opts
def link(url,text=None):
return "[{text}]({url})".format(url=url,text=url if text is None else text)
def reformat_to_markdown(desc):
def _indent4(mo):
i = " "
return i + mo.group(1).replace("\n",i)
def _repl_mention(mo):
return "#" + person_map[mo.group(1)]
#desc = desc.replace("\r","")
desc = re.sub("{noformat}(.+?){noformat}",_indent4,desc,flags=re.DOTALL+re.MULTILINE)
desc = re.sub(opts.jira_project+r"-(\d+)",r"issue #\1",desc)
desc = re.sub(r"\[~([^]]+)\]",_repl_mention,desc)
return desc
def fetch_issues(opts,jcl):
jql = [ 'project = ' + opts.jira_project ]
if opts.jira_components:
jql += [ ' OR '.join([ 'component = '+c for c in opts.jira_components ]) ]
if opts.jira_issues:
jql += [ ' OR '.join([ 'issue = '+i for i in opts.jira_issues ]) ]
jql_str = ' AND '.join(["("+q+")" for q in jql])
logging.info('executing query ' + jql_str)
return jcl.search_issues(jql_str,maxResults=500)
def jira_issue_to_bb_issue(opts,jcl,ji):
"""convert a jira issue to a dictionary with values appropriate for
POSTing as a bitbucket issue"""
logger = logging.getLogger(__name__)
content = reformat_to_markdown(ji.fields.description) if ji.fields.description else ''
if ji.fields.assignee is None:
resp = None
else:
resp = person_map[ji.fields.assignee.name]
reporter = person_map[ji.fields.reporter.name]
jiw = jcl.watchers(ji.key)
watchers = [ person_map[u.name] for u in jiw.watchers ] if jiw else []
milestone = None
if ji.fields.fixVersions:
vnames = [ v.name for v in ji.fields.fixVersions ]
milestone = vnames[0]
if len(vnames) > 1:
logger.warn("{ji.key}: bitbucket issues may have only 1 milestone (JIRA fixVersion); using only first ({f}) and ignoring rest ({r})".format(
ji=ji, f=milestone, r=",".join(vnames[1:])))
issue_id = extract_issue_number(ji.key)
bbi = {
'status': status_map[ji.fields.status.name],
'priority': priority_map[ji.fields.priority.name],
'kind': issuetype_map[ji.fields.issuetype.name],
'content_updated_on': ji.fields.created,
'voters': [],
'title': ji.fields.summary,
'reporter': reporter,
'component': None,
'watchers': watchers,
'content': content,
'assignee': resp,
'created_on': ji.fields.created,
'version': None, # ?
'edited_on': None,
'milestone': milestone,
'updated_on': ji.fields.updated,
'id': issue_id,
}
return bbi
def jira_comment_to_bb_comment(opts,jcl,jc):
bbc = {
'content': reformat_to_markdown(jc.body),
'created_on': jc.created,
'id': int(jc.id),
'updated_on': jc.updated,
'user': person_map[jc.author.name],
}
return bbc
def extract_issue_number(jira_issue_key):
return int(jira_issue_key.split('-')[-1])
def jira_key_to_bb_issue_tag(jira_issue_key):
return 'issue #' + str(extract_issue_number(jira_issue_key))
def jira_link_text(jk):
return link("https://invitae.jira.com/browse/"+jk,jk) + " (Invitae access required)"
if __name__ == '__main__':
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
opts = parse_args(sys.argv[1:])
dir_name = opts.jira_project
if opts.jira_components:
dir_name += '-' + ','.join(opts.jira_components)
if opts.jira_issues_file:
issues = [i.strip() for i in open(opts.jira_issues_file,'r')]
logger.info("added {n} issues from {opts.jira_issues_file} to issues list".format(n=len(issues),opts=opts))
opts.jira_issues += issues
opts.dir = os.path.join('/','tmp',dir_name)
opts.att_rel_dir = 'attachments'
opts.att_abs_dir = os.path.join(opts.dir,opts.att_rel_dir)
opts.json_fn = os.path.join(opts.dir,'db-1.0.json')
if not os.path.isdir(opts.att_abs_dir):
os.makedirs(opts.att_abs_dir)
opts.jira_issues = list(set(opts.jira_issues)) # distinctify
jcl = JIRA({'server': 'https://{opts.jira_hostname}/'.format(opts=opts)},
basic_auth=(opts.jira_username,opts.jira_password))
if opts.existing:
issues_db = json.load(open(opts.json_fn,'r'))
existing_ids = [ i['id'] for i in issues_db['issues'] ]
logger.info("read {n} issues from {fn}".format(n=len(existing_ids),fn=opts.json_fn))
else:
issues_db = dict()
issues_db['meta'] = {
'default_milestone': None,
'default_assignee': None,
'default_kind': "bug",
'default_component': None,
'default_version': None,
}
issues_db['attachments'] = []
issues_db['comments'] = []
issues_db['issues'] = []
issues_db['logs'] = []
issues_db['components'] = [ {'name':v.name} for v in jcl.project_components(opts.jira_project) ]
issues_db['milestones'] = [ {'name':v.name} for v in jcl.project_versions(opts.jira_project) ]
issues_db['versions'] = issues_db['milestones']
# bb_issue_map: bb issue # -> bitbucket issue
bb_issue_map = ImmutableDict( (i['id'],i) for i in issues_db['issues'] )
# jk_issue_map: jira key -> bitbucket issue
# contains only items migrated from JIRA (i.e., not preexisting issues with --existing)
jk_issue_map = ImmutableDict()
# issue_links is a dict of dicts of lists, using JIRA keys
# e.g., links['CORE-135']['depends on'] = ['CORE-137']
issue_links = collections.defaultdict(lambda: collections.defaultdict(lambda: []))
issues = fetch_issues(opts,jcl)
logger.info("fetch {n} issues from JIRA".format(n=len(issues)))
for ji in issues:
# Pfft. Need to fetch the issue again due to bug in JIRA.
# See https://bitbucket.org/bspeakmon/jira-python/issue/47/, comment on 2013-10-01 by ssonic
ji = jcl.issue(ji.key,expand="attachments,comments")
# create the issue
bbi = jira_issue_to_bb_issue(opts,jcl,ji)
issues_db['issues'] += [bbi]
bb_issue_map[bbi['id']] = bbi
jk_issue_map[ji.key] = bbi
issue_links[ji.key]['imported from'] = [jira_link_text(ji.key)]
# add comments
for jc in ji.fields.comment.comments:
bbc = jira_comment_to_bb_comment(opts,jcl,jc)
bbc['issue'] = bbi['id']
issues_db['comments'] += [bbc]
# add attachments
for ja in ji.fields.attachment:
att_rel_path = os.path.join(opts.att_rel_dir,ja.id)
att_abs_path = os.path.join(opts.att_abs_dir,ja.id)
if not os.path.exists(att_abs_path):
open(att_abs_path,'w').write(ja.get())
logger.info("Wrote {att_abs_path}".format(att_abs_path=att_abs_path))
bba = {
"path": att_rel_path,
"issue": bbi['id'],
"user": person_map[ja.author.name],
"filename": ja.filename,
}
issues_db['attachments'] += [bba]
# parent-child is task-subtask
if hasattr(ji.fields,'parent'):
issue_links[ji.fields.parent.key]['subtasks'].append(jira_key_to_bb_issue_tag(ji.key))
issue_links[ji.key]['parent task'].append(jira_key_to_bb_issue_tag(ji.fields.parent.key))
# add links
for il in ji.fields.issuelinks:
if hasattr(il,'outwardIssue'):
issue_links[ji.key][il.type.outward].append(jira_key_to_bb_issue_tag(il.outwardIssue.key))
elif hasattr(il,'inwardIssue'):
issue_links[ji.key][il.type.inward].append(jira_key_to_bb_issue_tag(il.inwardIssue.key))
logger.info("migrated issue {ji.key}: {ji.fields.summary} ({components})".format(
ji=ji,components=','.join(c.name for c in ji.fields.components)))
# append links section to content
# this section shows both task-subtask and "issue link" relationships
for src,dstlinks in issue_links.iteritems():
if src not in jk_issue_map:
logger.warn("issue {src}, with issue_links, not in jk_issue_map; skipping".format(src=src))
continue
links_block = "Links\n=====\n"
for desc,dsts in sorted(dstlinks.iteritems()):
links_block += "* **{desc}**: {links} \n".format(desc=desc,links=", ".join(dsts))
if jk_issue_map[src]['content']:
jk_issue_map[src]['content'] += "\n\n" + links_block
else:
jk_issue_map[src]['content'] = links_block
id_counts = collections.Counter(i['id'] for i in issues_db['issues'])
dupes = [ k for k,cnt in id_counts.iteritems() if cnt>1 ]
if dupes:
raise RuntimeError("{n} issue ids appear more than once from existing {opts.json_fn}".format(
n=len(dupes),opts=opts))
json.dump(issues_db,open(opts.json_fn,'w'))
logger.info("wrote {n} issues to {opts.json_fn}".format(n=len(id_counts),opts=opts))
# write zipfile
os.chdir(opts.dir)
with zipfile.ZipFile(opts.dir + '.zip','w') as zf:
for fn in ['db-1.0.json']+glob.glob('attachments/*'):
zf.write(fn)
logger.info("added {fn} to archive".format(fn=fn))
NOTE: I'm writing a new answer because writing this in a comment would be horrible, but most of the credit goes to #Turch's answer.
My steps (in OSX and Debian machines, both worked fine):
apt-get install python-pip (Debian) or sudo easy_install pip (OSX)
pip install jira
pip install configparser
easy_install -U setuptools (not sure if really needed)
Download or clone the source code from https://bitbucket.org/reece/rcore/ in your home folder, for example. Note: don't download using pip, it will get the 0.0.2 version and you need the 0.0.3.
Download the Python script created by Reece, mentioned by #Turch, and place it inside of the rcore folder.
Follow the instructions by #Turch: I also had to replace all instances of iteritems with items in the script and in rcore/types/immutabledict.py to make it work with Python 3. You will also need to fill in the dictionaries (priority_map, person_map, etc) with the values your project uses. Finally, you need a config file to exist with the connection info (see comments at the top of the script). Note: I used hostname like jira.domain.com (no http or https).
(This change did the trick for me) I had to change part of the line 250 from 'https://{opts.jira_hostname}/' to 'http://{opts.jira_hostname}/'
To finish, run the script like #Turch mentioned: The basic command line usage is export.py --jira-project <project>
The file was placed in /tmp/.zip for me.
The file was perfectly accepted in the BitBucket importer today.
Hooray for Reece and Turch! Thanks guys!
I'm trying to automate the deployment process, and as part of it, I need to run my release build from command line. I can do it, using command like
.\TFSBuild start http://server-name:8080/tfs/project-collection project-name build-name priority:High /queue
It even returns some code for the queued build — Build queued. Queue position: 2, Queue ID: 11057.
What I don't know, is how to get info about currently running builds, or about the state of my running build from powershell command line? The final aim is to start publishing after that build completes.
I've already got all necessary powershell scripts to create the deployment package from the build results, zip it, copy to production and install there. All I need now — to know when my build succeedes.
This function will wait for a build with the Queue ID given by TFSBuild.exe:
function Wait-QueuedBuild {
param(
$QueueID
)
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Build.Client')
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Client')
$uri = [URI]"http://server-name:8080/tfs/project-collection"
$projectCollection = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($uri)
$buildServer = $projectCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$spec = $buildServer.CreateBuildQueueSpec('*','*')
do {
$build = $buildServer.QueryQueuedBuilds($spec).QueuedBuilds| where {$_.Id -eq $QueueID}
sleep 1
} while ($build)
}
You can get the id returned by TFSBuild.exe, then call the function.
$tfsBuild = .\TFSBuild start http://server-name:8080/tfs/project-collection project-name build-name priority:High /queue
Wait-QueuedBuild [regex]::Match($tfsBuild[-1],'Queue ID: (?<id>\d+)').Groups['id'].Value
Using the work by E.Hofman available here it is possible to write a C# console app that uses TFS SDK and reveals if any build agent is currently running as follows:
using System;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
namespace ListAgentStatus
{
class Program
{
static void Main()
{
TfsTeamProjectCollection teamProjectCollection = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri("http://TFSServer:8080"));
var buildServer = teamProjectCollection.GetService<IBuildServer>();
foreach (IBuildController controller in buildServer.QueryBuildControllers(true))
{
foreach (IBuildAgent agent in controller.Agents)
{
Console.WriteLine(agent.Name+" is "+agent.IsReserved);
}
}
}
}
}
The parameter .IsReserved is what toggles to 'True' during execution of a build.
I 'm sorry my powershell skills are not good enough for providing with a PS variant of the above. Please take a look here, where the work by bwerks might help you do that.
# load classes for execution
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client") | Out-Null
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client") | Out-Null
# declare working variables
$Uri = New-Object System.Uri "http://example:8080/tfs"
# get reference to projection collection
$ProjectCollection = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($Uri)
# get reference to build server
$BuildServer = $ProjectCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
# loop through the build servers
foreach($Controller in $BuildServer.QueryBuildControllers($true))
{
# loop through agents
foreach($BuildAgent in $Controller.Agents)
{
Write-Host "$($BuildAgent.Name) is $($BuildAgent.IsReserved)"
}
}