programmatically export grafana dashboard data - grafana

I have a visual in grafana. I can manually go to the menu click export and export the time series data in json. This works great. Is there a way I can script that in python?. Is there some api I can hit that will return the json of a visual?
I was googling around and it looks like I can use the api to create dashboards/visuals and administer them but not sure where how to use the api to export the data.

Here's a Python script to export then dashboard json, not the presented data. Tested on Python 2.7:
#!/usr/bin/env python
"""Grafana dashboard exporter"""
import json
import os
import requests
HOST = 'http://localhost:3000'
API_KEY = os.environ["grafana_api_key"]
DIR = 'exported-dashboards/'
def main():
headers = {'Authorization': 'Bearer %s' % (API_KEY,)}
response = requests.get('%s/api/search?query=&' % (HOST,), headers=headers)
response.raise_for_status()
dashboards = response.json()
if not os.path.exists(DIR):
os.makedirs(DIR)
for d in dashboards:
print ("Saving: " + d['title'])
response = requests.get('%s/api/dashboards/%s' % (HOST, d['uri']), headers=headers)
data = response.json()['dashboard']
dash = json.dumps(data, sort_keys=True, indent=4, separators=(',', ': '))
name = data['title'].replace(' ', '_').replace('/', '_').replace(':', '').replace('[', '').replace(']', '')
tmp = open(DIR + name + '.json', 'w')
tmp.write(dash)
tmp.write('\n')
tmp.close()
if __name__ == '__main__':
main()
Usage:
You should first create an API key in Grafana and then run:
grafana_api_key=my-key python export-dash.py
Credit: This is a simplified version of https://github.com/percona/grafana-dashboards/blob/master/misc/export-dash.py

http://docs.grafana.org/http_api/data_source/#data-source-proxy-calls.
Visit your browser console (network tab) and you will see how it works there.

You could also use this Go client https://github.com/netsage-project/grafana-dashboard-manager
Its purpose is not what you are looking for, but it is possible to reuse that code.

Related

Linkedin API request dateRange end today or now

I'm using Python 3 to request share statistics from Linkedin API for the last 14 months.
It works fine, when I hardcoded the epoch values in the request link:
https://api.linkedin.com/v2/organizationalEntityShareStatistics?q=organizationalEntity&organizationalEntity=urn%3Ali%3Aorganization%3AXXXXX&timeIntervals=(timeRange:(start:1596206929000,end:1632938933000),timeGranularityType:DAY)
Obviously I don't want to be changing the end values each time I make a request, so I thought I'd declare a variable:
from datetime import datetime
epochcdt = datetime.now().timestamp()
And then just use it in the link instead of the hardcoded value:
https://api.linkedin.com/v2/organizationalEntityShareStatistics?q=organizationalEntity&organizationalEntity=urn%3Ali%3Aorganization%3AXXXXX&timeIntervals=(timeRange:(start:1596206929000,end:epochcdt),timeGranularityType:DAY)
But, that doesn't work:
{'message': 'Internal Server Error', 'status': 500}
Can you please help me with this hopefully, easy to solve problem?
Whole code:
import requests
import json
from liapiauth import auth, headers
from datetime import datetime
epochcdt = (datetime.now().timestamp())*1000
def organization_info(headers):
response = requests.get('https://api.linkedin.com/v2/organizationalEntityShareStatistics?q=organizationalEntity&organizationalEntity=urn%3Ali%3Aorganization%3AXXXXX&timeIntervals=(timeRange:(start:1596206929000,end:{epochcdt}),timeGranularityType:DAY)', headers = headers)
organization_info = response.json()
return organization_info
if __name__ == '__main__':
credentials = 'credentials.json'
access_token = auth(credentials)
headers = headers(access_token)
organization_info = organization_info(headers)
with open('lishare14m.json', 'w') as outfile:
json.dump(organization_info, outfile)
print(organization_info)

clients = self.AVAILABLE_CLIENTS[name] KeyError: 'requests' flask authlib client

good day everybody,
having some issues with flask and authlib. Bellow snip of my flash code
from flask import Flask, render_template
from authlib.integrations.flask_client import OAuth
import os
app = Flask(__name__)
app._static_folder = os.path.abspath("static")
app.config.from_object('config')
oauth = OAuth(app)
webex = oauth.register(name='WEBEX', redirect_uri='http://webapp.dcloud.cisco.com:5000/AuthWebex', client_kwargs={
'scope': 'spark:all'
} )
config.py
import os
WEBEX_CLIENT_ID='C3a256be511cdf07e19f272960c44a214aec14b727b108e4f10bd124d31d2112c'
WEBEX_CLIENT_SECRET='secret'
WEBEX_ACCESS_TOKEN_URL='https://api.ciscospark.com/v1/access_token'
WEBEX_REDIRECT_URI='http://localhost:5000/AuthWebex'
WEBEX_SCOPE='spark:all'
when running above code I get the following error:
File "/Users/tneumann/PycharmProjects/untitled/venv/lib/python3.7/site-packages/authlib/integrations/flask_client/oauth_registry.py", line 61, in register
self.use_oauth_clients()
File "/Users/tneumann/PycharmProjects/untitled/venv/lib/python3.7/site-packages/authlib/integrations/_client/oauth_registry.py", line 49, in use_oauth_clients
clients = self.AVAILABLE_CLIENTS[name]
KeyError: 'requests'
looked at examples and did some research, no luck. Can't find any solution...
thanks in adv.
Tobi
UPDATE:
per comment bellow here the latest code:
from flask import Flask, render_template, url_for, request
from authlib.integrations.flask_client import OAuth
import os
import requests
app = Flask(__name__)
app._static_folder = os.path.abspath("static")
app.config.from_object('config')
app.secret_key = os.urandom(24)
oauth = OAuth(app)
oauth.register(
'webex',
api_base_url='https://api.ciscospark.com/v1',
authorize_url='https://api.ciscospark.com/v1/authorize',
access_token_url='https://api.ciscospark.com/v1/access_token',
redirect_uri='http://webapp.dcloud.cisco.com:5000/AuthWebex',
scope='spark:all')
#app.route('/')
def main():
"""Entry point; the view for the main page"""
return render_template('main.html')
#app.route('/authorize')
def authorize():
return render_template('authorize.html')
#app.route('/login')
def login():
#redirect_uri = url_for('AuthWebex', _external=True)
redirect_uri = 'http://webapp.dcloud.cisco.com:5000/AuthWebex'
print(redirect_uri)
return oauth.webex.authorize_redirect(redirect_uri)
#app.route('/AuthWebex')
def AuthWebex():
#print(request.__dict__)
token = oauth.webex.authorize_access_token( authorization_response=request.url,
redirect_uri='http://webapp.dcloud.cisco.com:5000/AuthWebex',
client_id='C3a256be511cdf07e19f272960c44a214aec14b727b108e4f10bd124d31d2112c',
client_secret='secret',
)
print("Token: ", token)
resp = oauth.webex.get('https://api.ciscospark.com/v1/people/me')
profile = resp.json()
print(profile)
# do something with the token and profile
return '<html><body><h1>Authenticated</h1></body></html>'
if __name__ == '__main__':
app.run()
oauth.webex.authorize_access_token function throws and error when called without the parameters. which is strange as most examples I found exactly do that.
client_id and client_secret are set via the config.py file. This works for the oauth.register function but not for the authorize_access_token.
Additional problem is that even with the parameters, it produces a valid token. When I call the get function I get the following error:
File "/Users/tneumann/PycharmProjects/untitled/venv/lib/python3.7/site-packages/requests/models.py", line 317, in prepare
self.prepare_auth(auth, url)
File "/Users/tneumann/PycharmProjects/untitled/venv/lib/python3.7/site-packages/requests/models.py", line 548, in prepare_auth
r = auth(self)
File "/Users/tneumann/PycharmProjects/untitled/venv/lib/python3.7/site-packages/authlib/integrations/requests_client/oauth2_session.py", line 41, in __call__
raise UnsupportedTokenTypeError(description=description)
authlib.integrations._client.errors.UnsupportedTokenTypeError: unsupported_token_type: Unsupported token_type: 'token_type'
here is the format of the token returned from authorize_access_token function
Token: {'access_token': 'YWIzNGU3<secret>tNDQ5_PF84_7cc07dbd-<secret>-5877334424fd', 'expires_in': 1209599, 'refresh_token': 'MjU2ZDM4N2Et<secret>ZmItMTg5_PF84_7cc07dbd-<secret>877334424fd', 'refresh_token_expires_in': 7722014, 'expires_at': 1574863645}
went through the docs, the code on github and debugging in pycharm with no luck, help would be much appreciated!
The problem here is that this AuthWebex is not a standard OAuth service. The response has no token_type. We can fix it with Authlib compliance fix:
Check the example here:
https://docs.authlib.org/en/latest/client/frameworks.html#compliance-fix-for-oauth-2-0
The slack example has the same issue.

Watson Discovery Service Python add document Error: Invalid Content-Type. Expected 'multipart/form-data'

Import Watson Developer Cloud Python SDK
from watson_developer_cloud import DiscoveryV1
Get the pdf from Slack doc_url which is the private URL
r = requests.get(doc_url, headers={'Authorization': 'Bearer {}'.format(slack_token) })
logging.debug("read_pdf headers %s " %r.headers )
logging.debug("read_pdf content-type %s " %r.headers['content-type'] )
Save the file in the cloud file system temporarily
with open(doc_name, 'wb' ) as f:
f.write(r.content)
filepath = os.path.join(os.getcwd(), '.', doc_name )
logging.debug('filepath %s' %filepath)
logging.debug('filepath assertion %s' %os.path.isfile(filepath) )
Create a Discovery instance
discovery = DiscoveryV1(
username=DS_USERNAME,
password=DS_PASSWORD,
version="2017-10-16"
)
Add pdf document in Discovery instance
with open(filepath, 'rb') as fileinfo:
add_doc = discovery.add_document(ENVIRONMENT_ID, COLLECTION_ID, file_content_type=r.headers['content-type'])
Log files
read_pdf headers {'Content-Type': 'application/pdf', 'Content-Length': '149814'
WatsonApiException: Error: Invalid Content-Type. Expected 'multipart/form-data', got 'application/octet-stream', Code: 400 , X-dp-watson-tran-id: gateway02-732476861 , X-global-transaction-id: ffea405d5ba1ad632ba8b5bd
Developer code examples are commented out in Github.
https://github.com/watson-developer-cloud/python-sdk/blob/master/examples/discovery_v1.py
Oh, my. That is a miserable error message.
What is missing from the call to discovery.add_document() is the file parameter. Can you try adding file=fileinfo like this:
with open(filepath, 'rb') as fileinfo:
add_doc = discovery.add_document(ENVIRONMENT_ID,
COLLECTION_ID,
file=fileinfo,
file_content_type=r.headers['content-type'])
For reference, here is some Python code that works and is doing something very similar to what it looks like you are aiming for.

What's going wrong when I try to create a review comment through Github's v3 API?

I'm trying to create a review commit through Github's v3 API and am not succeeding. Consider this repository. There's a single pull request and for the purposes of this question let's say I want to leave a 'changes requested' review on that PR. Here's the code I've got:
#!/usr/bin/env python3
import requests
import json
TOKEN='YOUR_TOKEN_HERE'
REPO = "blt/experiment-repo"
PR_NUM = 1
COMMIT_SHA_1 = "4160bee478c3c985eaaa35f161cc922fe20b354a"
COMMIT_SHA_2 = "df9d13a2e35f9b6c228e1f30ea30585ed85af26a"
def main():
pr_comment_headers = {
'user-agent': 'benedikt/0.0.1',
'Authorization': 'token %s' % TOKEN,
# Accept header per
# https://developer.github.com/changes/2016-12-16-review-requests-api/
'Accept': 'application/vnd.github.black-cat-preview+json',
}
msg = "BLEEP BLOOP I AM A ROBOT"
payload = { 'commit_id': COMMIT_SHA_2,
'body': msg,
'event': "REQUEST_CHANGES" }
# Per https://developer.github.com/v3/pulls/reviews/#create-a-pull-request-review
review_url = "https://api.github.com/repos/%s/pulls/%s/reviews" % (REPO, PR_NUM)
res = requests.post(review_url, headers = pr_comment_headers,
json = json.dumps(payload))
print(res)
print(res.text)
if __name__ == '__main__':
main()
I've marked in code comments where I've discovered the API endpoints to hit and with what payloads. Excepting, I must have goofed somewhere because when I run the above program I receive:
<Response [422]>
{"message":"Validation Failed","errors":["Variable commitOID of type GitObjectID was provided invalid value","Variable event of type PullRequestReviewEvent was provided invalid value"],"documentation_url":"https://developer.github.com/v3/pulls/reviews/#create-a-pull-request-review"}
I've verified that the commit SHAs are the exact ones that Github shows and REQUEST_CHANGES is the string in the documentation.
What am I missing?
I think you need to let requests encode the request body instead of encoding it yourself with json.dumps(), something like this: requests.post(..., json=payload)

How can I export Jira issues to BitBucket

Ive just moved my projects code from java.net to BitBucket. But my jira issue tracking is still hosted on java.net, although BitBucket does have some options for linking to an external issue tracker I don't think I can use it for java.net, not least because I do not have the admin priviledges need to install the DVCS connector.
So I thought an alternative option would be to export and then import the issues into BitBucket issue tracker, is that possible ?
Progress so far
So I tried following the steps in both informative answers using OSX below but I hit a problem - I'm rather confused about what the script would actually be called because in the answers it talks about export.py but no such script exists with that name so I renamed the one I downloaded.
sudo easy_install pip (OSX)
pip install jira
pip install configparser
easy_install -U setuptools
Go to https://bitbucket.org/reece/rcore, select downloads tab, download zip and unzip, and rename to reece ( for some reason git clone https://bitbucket.org/reece/rcore fails with error)
cd reece/rcore
Save script as export.py in rcore subfolder
Replace iteritems with items in import.py
Replace iteritems with types/immutabledict.py
Create .config in rcore folder
Create .config/jira-issues-move-to-bitbucket.conf containing
jira-username=paultaylor
jira-hostname=https://java.net/jira/browse/JAUDIOTAGGER
jira-password=password
Run python export.py --jira-project jaudiotagger
gives
macbook:rcore paul$ python export.py --jira-project jaudiotagger
Traceback (most recent call last):
File "export.py", line 24, in <module>
import configparser
ImportError: No module named configparser
- Run python export.py --jira-project jaudiotagger
I need to run pip insdtall as root so did
sudo pip install configparser
and that worked
but now
python export.py --jira.project jaudiotagger
gives
File "export.py" line 35, in <module?
from jira.client import JIRA
ImportError: No module named jira.client
You can import issues into BitBucket, they just need to be in the appropriate format. Fortunately, Reece Hart has already written a Python script to connect to a Jira instance and export the issues.
To get the script to run I had to install the Jira Python package as well as the latest version of rcore (if you use pip you get an incompatible previous version, so you have to get the source). I also had to replace all instances of iteritems with items in the script and in rcore/types/immutabledict.py to make it work with Python 3. You will also need to fill in the dictionaries (priority_map, person_map, etc) with the values your project uses. Finally, you need a config file to exist with the connection info (see comments at the top of the script).
The basic command line usage is export.py --jira-project <project>
Once you've got the data exported, see the instructions for importing issues to BitBucket
#!/usr/bin/env python
"""extract issues from JIRA and export to a bitbucket archive
See:
https://confluence.atlassian.com/pages/viewpage.action?pageId=330796872
https://confluence.atlassian.com/display/BITBUCKET/Mark+up+comments
https://bitbucket.org/tutorials/markdowndemo/overview
2014-04-12 08:26 Reece Hart <reecehart#gmail.com>
Requires a file ~/.config/jira-issues-move-to-bitbucket.conf
with content like
[default]
jira-username=some.user
jira-hostname=somewhere.jira.com
jira-password=ur$pass
"""
import argparse
import collections
import configparser
import glob
import itertools
import json
import logging
import os
import pprint
import re
import sys
import zipfile
from jira.client import JIRA
from rcore.types.immutabledict import ImmutableDict
priority_map = {
'Critical (P1)': 'critical',
'Major (P2)': 'major',
'Minor (P3)': 'minor',
'Nice (P4)': 'trivial',
}
person_map = {
'reece.hart': 'reece',
# etc
}
issuetype_map = {
'Improvement': 'enhancement',
'New Feature': 'enhancement',
'Bug': 'bug',
'Technical task': 'task',
'Task': 'task',
}
status_map = {
'Closed': 'resolved',
'Duplicate': 'duplicate',
'In Progress': 'open',
'Open': 'new',
'Reopened': 'open',
'Resolved': 'resolved',
}
def parse_args(argv):
def sep_and_flatten(l):
# split comma-sep elements and flatten list
# e.g., ['a','b','c,d'] -> set('a','b','c','d')
return list( itertools.chain.from_iterable(e.split(',') for e in l) )
cf = configparser.ConfigParser()
cf.readfp(open(os.path.expanduser('~/.config/jira-issues-move-to-bitbucket.conf'),'r'))
ap = argparse.ArgumentParser(
description = __doc__
)
ap.add_argument(
'--jira-hostname', '-H',
default = cf.get('default','jira-hostname',fallback=None),
help = 'host name of Jira instances (used for url like https://hostname/, e.g., "instancename.jira.com")',
)
ap.add_argument(
'--jira-username', '-u',
default = cf.get('default','jira-username',fallback=None),
)
ap.add_argument(
'--jira-password', '-p',
default = cf.get('default','jira-password',fallback=None),
)
ap.add_argument(
'--jira-project', '-j',
required = True,
help = 'project key (e.g., JRA)',
)
ap.add_argument(
'--jira-issues', '-i',
action = 'append',
default = [],
help = 'issue id (e.g., JRA-9); multiple and comma-separated okay; default = all in project',
)
ap.add_argument(
'--jira-issues-file', '-I',
help = 'file containing issue ids (e.g., JRA-9)'
)
ap.add_argument(
'--jira-components', '-c',
action = 'append',
default = [],
help = 'components criterion; multiple and comma-separated okay; default = all in project',
)
ap.add_argument(
'--existing', '-e',
action = 'store_true',
default = False,
help = 'read existing archive (from export) and merge new issues'
)
opts = ap.parse_args(argv)
opts.jira_components = sep_and_flatten(opts.jira_components)
opts.jira_issues = sep_and_flatten(opts.jira_issues)
return opts
def link(url,text=None):
return "[{text}]({url})".format(url=url,text=url if text is None else text)
def reformat_to_markdown(desc):
def _indent4(mo):
i = " "
return i + mo.group(1).replace("\n",i)
def _repl_mention(mo):
return "#" + person_map[mo.group(1)]
#desc = desc.replace("\r","")
desc = re.sub("{noformat}(.+?){noformat}",_indent4,desc,flags=re.DOTALL+re.MULTILINE)
desc = re.sub(opts.jira_project+r"-(\d+)",r"issue #\1",desc)
desc = re.sub(r"\[~([^]]+)\]",_repl_mention,desc)
return desc
def fetch_issues(opts,jcl):
jql = [ 'project = ' + opts.jira_project ]
if opts.jira_components:
jql += [ ' OR '.join([ 'component = '+c for c in opts.jira_components ]) ]
if opts.jira_issues:
jql += [ ' OR '.join([ 'issue = '+i for i in opts.jira_issues ]) ]
jql_str = ' AND '.join(["("+q+")" for q in jql])
logging.info('executing query ' + jql_str)
return jcl.search_issues(jql_str,maxResults=500)
def jira_issue_to_bb_issue(opts,jcl,ji):
"""convert a jira issue to a dictionary with values appropriate for
POSTing as a bitbucket issue"""
logger = logging.getLogger(__name__)
content = reformat_to_markdown(ji.fields.description) if ji.fields.description else ''
if ji.fields.assignee is None:
resp = None
else:
resp = person_map[ji.fields.assignee.name]
reporter = person_map[ji.fields.reporter.name]
jiw = jcl.watchers(ji.key)
watchers = [ person_map[u.name] for u in jiw.watchers ] if jiw else []
milestone = None
if ji.fields.fixVersions:
vnames = [ v.name for v in ji.fields.fixVersions ]
milestone = vnames[0]
if len(vnames) > 1:
logger.warn("{ji.key}: bitbucket issues may have only 1 milestone (JIRA fixVersion); using only first ({f}) and ignoring rest ({r})".format(
ji=ji, f=milestone, r=",".join(vnames[1:])))
issue_id = extract_issue_number(ji.key)
bbi = {
'status': status_map[ji.fields.status.name],
'priority': priority_map[ji.fields.priority.name],
'kind': issuetype_map[ji.fields.issuetype.name],
'content_updated_on': ji.fields.created,
'voters': [],
'title': ji.fields.summary,
'reporter': reporter,
'component': None,
'watchers': watchers,
'content': content,
'assignee': resp,
'created_on': ji.fields.created,
'version': None, # ?
'edited_on': None,
'milestone': milestone,
'updated_on': ji.fields.updated,
'id': issue_id,
}
return bbi
def jira_comment_to_bb_comment(opts,jcl,jc):
bbc = {
'content': reformat_to_markdown(jc.body),
'created_on': jc.created,
'id': int(jc.id),
'updated_on': jc.updated,
'user': person_map[jc.author.name],
}
return bbc
def extract_issue_number(jira_issue_key):
return int(jira_issue_key.split('-')[-1])
def jira_key_to_bb_issue_tag(jira_issue_key):
return 'issue #' + str(extract_issue_number(jira_issue_key))
def jira_link_text(jk):
return link("https://invitae.jira.com/browse/"+jk,jk) + " (Invitae access required)"
if __name__ == '__main__':
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
opts = parse_args(sys.argv[1:])
dir_name = opts.jira_project
if opts.jira_components:
dir_name += '-' + ','.join(opts.jira_components)
if opts.jira_issues_file:
issues = [i.strip() for i in open(opts.jira_issues_file,'r')]
logger.info("added {n} issues from {opts.jira_issues_file} to issues list".format(n=len(issues),opts=opts))
opts.jira_issues += issues
opts.dir = os.path.join('/','tmp',dir_name)
opts.att_rel_dir = 'attachments'
opts.att_abs_dir = os.path.join(opts.dir,opts.att_rel_dir)
opts.json_fn = os.path.join(opts.dir,'db-1.0.json')
if not os.path.isdir(opts.att_abs_dir):
os.makedirs(opts.att_abs_dir)
opts.jira_issues = list(set(opts.jira_issues)) # distinctify
jcl = JIRA({'server': 'https://{opts.jira_hostname}/'.format(opts=opts)},
basic_auth=(opts.jira_username,opts.jira_password))
if opts.existing:
issues_db = json.load(open(opts.json_fn,'r'))
existing_ids = [ i['id'] for i in issues_db['issues'] ]
logger.info("read {n} issues from {fn}".format(n=len(existing_ids),fn=opts.json_fn))
else:
issues_db = dict()
issues_db['meta'] = {
'default_milestone': None,
'default_assignee': None,
'default_kind': "bug",
'default_component': None,
'default_version': None,
}
issues_db['attachments'] = []
issues_db['comments'] = []
issues_db['issues'] = []
issues_db['logs'] = []
issues_db['components'] = [ {'name':v.name} for v in jcl.project_components(opts.jira_project) ]
issues_db['milestones'] = [ {'name':v.name} for v in jcl.project_versions(opts.jira_project) ]
issues_db['versions'] = issues_db['milestones']
# bb_issue_map: bb issue # -> bitbucket issue
bb_issue_map = ImmutableDict( (i['id'],i) for i in issues_db['issues'] )
# jk_issue_map: jira key -> bitbucket issue
# contains only items migrated from JIRA (i.e., not preexisting issues with --existing)
jk_issue_map = ImmutableDict()
# issue_links is a dict of dicts of lists, using JIRA keys
# e.g., links['CORE-135']['depends on'] = ['CORE-137']
issue_links = collections.defaultdict(lambda: collections.defaultdict(lambda: []))
issues = fetch_issues(opts,jcl)
logger.info("fetch {n} issues from JIRA".format(n=len(issues)))
for ji in issues:
# Pfft. Need to fetch the issue again due to bug in JIRA.
# See https://bitbucket.org/bspeakmon/jira-python/issue/47/, comment on 2013-10-01 by ssonic
ji = jcl.issue(ji.key,expand="attachments,comments")
# create the issue
bbi = jira_issue_to_bb_issue(opts,jcl,ji)
issues_db['issues'] += [bbi]
bb_issue_map[bbi['id']] = bbi
jk_issue_map[ji.key] = bbi
issue_links[ji.key]['imported from'] = [jira_link_text(ji.key)]
# add comments
for jc in ji.fields.comment.comments:
bbc = jira_comment_to_bb_comment(opts,jcl,jc)
bbc['issue'] = bbi['id']
issues_db['comments'] += [bbc]
# add attachments
for ja in ji.fields.attachment:
att_rel_path = os.path.join(opts.att_rel_dir,ja.id)
att_abs_path = os.path.join(opts.att_abs_dir,ja.id)
if not os.path.exists(att_abs_path):
open(att_abs_path,'w').write(ja.get())
logger.info("Wrote {att_abs_path}".format(att_abs_path=att_abs_path))
bba = {
"path": att_rel_path,
"issue": bbi['id'],
"user": person_map[ja.author.name],
"filename": ja.filename,
}
issues_db['attachments'] += [bba]
# parent-child is task-subtask
if hasattr(ji.fields,'parent'):
issue_links[ji.fields.parent.key]['subtasks'].append(jira_key_to_bb_issue_tag(ji.key))
issue_links[ji.key]['parent task'].append(jira_key_to_bb_issue_tag(ji.fields.parent.key))
# add links
for il in ji.fields.issuelinks:
if hasattr(il,'outwardIssue'):
issue_links[ji.key][il.type.outward].append(jira_key_to_bb_issue_tag(il.outwardIssue.key))
elif hasattr(il,'inwardIssue'):
issue_links[ji.key][il.type.inward].append(jira_key_to_bb_issue_tag(il.inwardIssue.key))
logger.info("migrated issue {ji.key}: {ji.fields.summary} ({components})".format(
ji=ji,components=','.join(c.name for c in ji.fields.components)))
# append links section to content
# this section shows both task-subtask and "issue link" relationships
for src,dstlinks in issue_links.iteritems():
if src not in jk_issue_map:
logger.warn("issue {src}, with issue_links, not in jk_issue_map; skipping".format(src=src))
continue
links_block = "Links\n=====\n"
for desc,dsts in sorted(dstlinks.iteritems()):
links_block += "* **{desc}**: {links} \n".format(desc=desc,links=", ".join(dsts))
if jk_issue_map[src]['content']:
jk_issue_map[src]['content'] += "\n\n" + links_block
else:
jk_issue_map[src]['content'] = links_block
id_counts = collections.Counter(i['id'] for i in issues_db['issues'])
dupes = [ k for k,cnt in id_counts.iteritems() if cnt>1 ]
if dupes:
raise RuntimeError("{n} issue ids appear more than once from existing {opts.json_fn}".format(
n=len(dupes),opts=opts))
json.dump(issues_db,open(opts.json_fn,'w'))
logger.info("wrote {n} issues to {opts.json_fn}".format(n=len(id_counts),opts=opts))
# write zipfile
os.chdir(opts.dir)
with zipfile.ZipFile(opts.dir + '.zip','w') as zf:
for fn in ['db-1.0.json']+glob.glob('attachments/*'):
zf.write(fn)
logger.info("added {fn} to archive".format(fn=fn))
NOTE: I'm writing a new answer because writing this in a comment would be horrible, but most of the credit goes to #Turch's answer.
My steps (in OSX and Debian machines, both worked fine):
apt-get install python-pip (Debian) or sudo easy_install pip (OSX)
pip install jira
pip install configparser
easy_install -U setuptools (not sure if really needed)
Download or clone the source code from https://bitbucket.org/reece/rcore/ in your home folder, for example. Note: don't download using pip, it will get the 0.0.2 version and you need the 0.0.3.
Download the Python script created by Reece, mentioned by #Turch, and place it inside of the rcore folder.
Follow the instructions by #Turch: I also had to replace all instances of iteritems with items in the script and in rcore/types/immutabledict.py to make it work with Python 3. You will also need to fill in the dictionaries (priority_map, person_map, etc) with the values your project uses. Finally, you need a config file to exist with the connection info (see comments at the top of the script). Note: I used hostname like jira.domain.com (no http or https).
(This change did the trick for me) I had to change part of the line 250 from 'https://{opts.jira_hostname}/' to 'http://{opts.jira_hostname}/'
To finish, run the script like #Turch mentioned: The basic command line usage is export.py --jira-project <project>
The file was placed in /tmp/.zip for me.
The file was perfectly accepted in the BitBucket importer today.
Hooray for Reece and Turch! Thanks guys!