ipywidgets, problem using a Text widget to filter options in a Select widget with automatic updates/ real-time interaction - select

I'm trying to use ipywidgets for the first time to make an interactive browser in Jupyter Notebooks of Census data using the census module. In general I'm having difficulty learning the widget library. My plan is to use the Text widget to enter a search term that will then filter the available census 'concepts' or table options (example: "Means of transportation to work") in a Selection or SelectMultiple widget. The selections made will then filter and show available data variables / columns for that census 'concept' or table (example: "car", "truck", "bus", "bike", "walk")
For reference, this first code block is what I got to work in a normal Jupyter Notebook with no widgets (at the bottom is the ipywidget attempt I'm having trouble with). I'm just starting to explore widgets and don't know anyone with experience with them. Any assistance is greatly appreciated!
*Note, running my code would require a census api key from census.gov/data/developers.html, the "Request a KEY" option on the left side menu, and change the ckey="" variable in the code below to your key. Keys are free, as it is publicly available government data.
#This section works for me as standalone code, not using ipywidgets
#Show all concepts to get an idea of search phrases to use
concepts_all = list([n for n in sorted(list([f"{t['description']}" for t in c.acs5.tables()]))]);
#This is a VERY long list, so I commented out printing it
#[print(c) for c in concepts_all]
#Apply a search phrase and limit options accordingly
concept_description_search_phrase = "Means of transportation to work by travel"
[print(f"{table['name']}: {table['description']}") for table in c.acs5.tables() if concept_description_search_phrase.lower() in table['description'].lower()]
#From the filtered options, chose one specific one to search, and display available data for it
concept_full_description = 'MEANS OF TRANSPORTATION TO WORK BY TRAVEL TIME TO WORK'
variable_filter_phrase = 'bicycle'
df = [pd.read_json(table['variables']) for table in c.acs5.tables() if table['description'] == concept_full_description][0]
acs_variable_names_list = [row.name for n,row in df.iterrows()
if variable_filter_phrase.lower() in row['variables']['label'].lower()
and 'Annotation' not in row['variables']['label']
and 'Margin of Error' not in row['variables']['label']]
acs_variable_names_list
# This is the final list I want out from a ipywidgets based workflow
# Which are variable columns, (ex: number of people in each census block group that
# have a 10, 15, 30, 45, 60+ minute commute by bicycle
########################################################
I'm having trouble getting the ipywidgets to work properly, where the Text box will interactively filter the available options in the Selection menu. I can only get it to work on a default value for the Text widget, but not off of the interactive menu. I'm basically going for the effect of the Combobox widget, but with the option to select more than one item if desired, and still see all the available options after a selection is made (so I'm thinking to use a Text and Select widget together instead of Combobox). This is my attempt so far:
out = widgets.Output()
display(out)
search_phrase = widgets.Text(
value = 'transportation',
layout = widgets.Layout(width = '900px')
)
select_multiple = widgets.SelectMultiple(
options = [n for n in concepts_all if search_phrase.value.lower() in n.lower()],
layout = widgets.Layout(width='900px',height='250px'),
ensure_option = True,
disabled=False
)
def search_change(change):
output.clear_output()
display(search_phrase)
with output:
if change.new == '':
print('new search')
return search_phrase.value
else:
print('same search')
return search_phrase.value
display(search_phrase)
display(select_multiple)
#Issue here with getting the search_phrase to properly filter options in select_multiple
#
concept_full_description = select_multiple.value[0] #Using Widgets
variable_filter_phrase = widgets.Text(value = 'bicy')
display(variable_filter_phrase) #Issue, this text box won't automatically filter the output
#it is only filtering by the default value
df = [pd.read_json(table['variables']) for table in c.acs5.tables() if table['description'] == concept_full_description][0]
acs_variable_names_list = [row.name for n,row in df.iterrows()
if variable_filter_phrase.value.lower() in row['variables']['label'].lower()
and 'Annotation' not in row['variables']['label']
and 'Margin of Error' not in row['variables']['label']] #print(row.name,": ",row['variables']['label'])
acs_variable_names_list
#This is the desired output list that will go into later workflows
I think I'm doing something wrong either in use of the .observe(), .interact(), or output /clear_output process. I'm just learning the ipywidgets and am even having trouble getting this to work in a basic example trying to follow the ipywidgets.readthedocs.io documentation.

Related

How to read info on voltage/beam energy, imaging mode, acquisition date/timestamp, etc. from image meta-data? (Tags)

DM scripting beginner here, almost no programming skills.
I would like to know the commands to access all the metadata of DM images/spectra.
I realized that all my STEM images at 80 kV taken between 2 dates (let's say 02.11.2017-05.04.2019) have the scale calibration wrong by the same factor (scale of all such images needs to be multiplied by 1.21).
I would like to write a script which multiplies the scale value by a factor only for images in scanning mode at 80 kV taken during a period for all images in a folder with subfolders or for all images opened in DM and save the new scale value.
I checked this website http://digitalmicrograph-scripting.tavernmaker.de/other%20resources/Old-DMHelp/AllFunctions.html but only found how to call the scale value (ImageGetDimensionCalibration). I have a general idea how to write the script based on other scripts if I find out how to call the metadata.
If anyone can write the whole script for me I would greatly appreciate your effort.
All general meta-data is organized in the image tag-structure
You can see this, if you open the Image Display Info of an image. (Via the menu, or by pressing CTRL + D) and then browse to the "Tags" section:
All info on the right are image tags and they are organized in a hierarchical tree.
How this tree looks like, and what information is written where, is totally open and will depend on what GMS version you are using, how the hardware is configured etc. Also custom scripts might alter this information.
So for a scripting start, open the data you want to modify and have a look in this tree.
Hint: The following min-script can be useful. It opens a tag-browsing window for the front-most image but as a modeless dialog (i.e. you can keep it open and interact with other parts):
GetFrontImage().ImageGetTagGroup().TagGroupOpenBrowserWindow(0)
The information you need to check against is most probably found in the Microscope Info sub-tree. Here, usually all information gathered from the microscope during acquisition is stored. What is there, will depend on your system and how it is set up.
The information of the STEM image acquisition - as far as the scanning engine and detector is concerned - is most probably in the DigiScan sub-tree.
The Data Bar sub-tree usually contains date and time of creation etc.
Calibration values are not stored in the image tag-structure
What you will not find in this tag-structure is the image calibration, i.e. the values actually used by DM to display calibrated values. These values are "one level up" so to speak here:
This is important to know in the following for your script, because you will need different commands for both the "meta-data" from the tags, and the "calibration" you want to change.
Accessing meta-data by script
The script-commands you need to read from the tags are all described in the F1 help documentation here:
Essentially, you need a command to get the "root" TagGroup of an image, which is ImageGetTagGroup() and then you traverse within this tree.
This might seem confusing - because there are a lot of slightly different commands for the different types of stored tags - but the essential bits are easy:
All "Paths" through the tree are just the individual names (typed exactly)
For each "branch" you have to use a single colon :
The commands to set/get a tag-value all require as input the "root" tagGroup object and the "path" as a string. The get commands require a variable of matching type to store the value in, the set commands need the value which should be written.
= The get commands themeselves return true or false depending on whether or not a tag-path could be found and the value could be read.
So the following script would read the "Imaging Mode" from the tags of the image shown as example above:
string mode
GetFrontImage().ImageGetTagGroup().TagGroupGetTagAsString( "Microscope Info:Imaging Mode", mode )
OKDialog( "Mode: " + mode )
and in a little more verbose form:
string mode // variable to hold the value
image img // variable for the image
string path // variable/constant to specify the where
TagGroup tg // variable to hold the "tagGroup" object
img := GetFrontImage() // Use the selected image
tg = img.ImageGetTagGroup() // From the image get the tags (root)
path = "Microscope Info:Imaging Mode" // specify the path
if ( tg.TagGroupGetTagAsString( path, mode ) )
OKDialog( "Mode: " + mode )
else
Throw( "Tag not found" )
If the tag is not a string but a value, you will need the according commands, i.e.
TagGroupGetTagAsNumber().

Error in Opening Report from Form

I have a Form which will help me to filter out the records I want for my Report. The button will open the Report On Click.
This is the code in the button:
Private Sub Open_OEE_Click()
DoCmd.OpenReport "OEE_Report", acViewReport, , , acWindowNormal
End Sub
I keep getting the error:
I also have placed the query in my report under the Record Source as:
SELECT * FROM 3_OEE WHERE ((([3_OEE].RecordID)=Forms![3_OEE_Report]!cboRecordID) And (([3_OEE].Date_Recorded)=DateValue(Forms![3_OEE_Report]!Date_Recorded)) And (([3_OEE].MC_No)=Forms![3_OEE_Report]!cboMCNo) And (([3_OEE].Product)=Forms![3_OEE_Report]!cboProduct));
I want to search based on one criteria (text box or combo box) and not all four at once.
Am I missing out something?
MS-Access does tend to go a bit overboard with the brackets. Make the report's Record Source a bit easier to read by trimming out the unnecessary ones. You also need to get your date criterion in the right format - Access always uses US formatting in SQL queries and needs # signs around the date:
SELECT * FROM 3_OEE
WHERE [3_OEE].RecordID = Forms![3_OEE_Report]!cboRecordID
And [3_OEE].Date_Recorded = Format(Forms![3_OEE_Report]!Date_Recorded, "\#mm/dd/yyyy\#")
And [3_OEE].MC_No = Forms![3_OEE_Report]!cboMCNo
And [3_OEE].Product = Forms![3_OEE_Report]!cboProduct;
I would also suggest creating a named query for this and setting the report's Record Source to the named query. You can then test the query in isolation without having to run the report (but make sure the Form is open and the relevant controls are populated).
I asked for help from another source.
Answer to Question

Creating custom widgets for ruby gem dashing.io - Or combining widgets elements

I'm working with the dashing ruby gem and I would really like to combine elements of the graph and number widgets. I would like all elements of the graph and include the up/down percentage arrow from the number widget.
I've never done much work with ruby and I understand there are a few files within the widget that might need to be changed.
I've setup a few nice widgets and I'm using a job to pull data from a redis db to populate. I've added the following to the graph.html page:
<p class="change-rate">
<i data-bind-class="arrow"></i><span data-bind="difference"></span>
</p>
This has no effect, and I'm sure that I'm missing something in one of the many files that makes this all work.
Your on the right track, and I've actually put together something very similar but in order to complete what you are trying to do you need to send data to your two new data binds, which would be done with your jobs file and the graph.coffee file.
I'm not sure how exactly you're getting your graph data from redis to your jobs erb file but you will want to setup a couple new variables, for the example I have used nowNumber and lastNumber. These will be the number that the valuation is performed on.
jobs/jobname.erb
send_event('graph', points: points, current: nowNumber, last: lastNumber )
If you print this out you will get something like this:
{:points=>[{:x=>6, :y=>64}, {:x=>5, :y=>62}, {:x=>4, :y=>56}], :current=>57, :last=>64}
Tweak your graph/graph.coffee file:
# The following 6 lines aren't needed for your solution, but if you wanted to take advantage of 'warning', 'danger', and 'ok' status classes (changes background color and flashes), just feed your send_event with 'status: [one of the three status names]
if data.status
# clear existing "status-*" classes
$(#get('node')).attr 'class', (i,c) ->
c.replace /\bstatus-\S+/g, ''
# add new class
$(#get('node')).addClass "status-#{data.status}"
#accessor 'difference', ->
if #get('last')
last = parseInt(#get('last'))
current = parseInt(#get('current'))
if last != 0
diff = Math.abs(Math.round((current - last) / last * 100))
"#{diff}%"
else
""
# Picks the direction of the arrow based on whether the current value is higher or lower than the last
#accessor 'arrow', ->
if #get('last')
if parseInt(#get('current')) > parseInt(#get('last')) then 'icon-arrow-up' else 'icon-arrow-down'

How can I do breaks and subtotals in a report?

I need to generate a business report using perl + Template Toookit and LaTeX.
Things are working really well, but I am struggling with the problem of having breaks (for example page breaks, or special headers) and subtotals whenever a field changes.
So, for example, every time the field "category" changes, I'd need to have a total of sales for that category, and a header showing that another category listing is starting; and then do the same when the field "group" - with the added interest that "group" is made up of categories, so the two things should nest.
I guess anyone that has built reports with Microsoft Access (or probably any other business reporting application) should be familiar with the problem.
Ideally this would be solved at a meta-level, so I don't have to rebuild the code every time, but only to specify what fields should generate breaks or subtotals.
I am (voluntarily) constrained to LaTeX and TT: LaTeX because of the control it gives over typography, and the possibility of generating custom graphics, and TT (or anything else that works in perl) because of learning curves.
There's no built-in subtotaling feature in TT, but you could possibly put your data into a Data::Table object, that would give you some ability to handle subtotaling at the 'meta' level, as you say.
Depending on the number of columns involved though, it might be just as simple to create local hashes to maintain running totals: NB: untested, example code only
[%-
MACRO printrow(rowtype, line) BLOCK;
# however you print the row as LaTeX
# rowtype is 'row', 'subtotal' or 'grandtotal' for formatting purposes
END;
SET sumcols = [ 'col3', 'col4', 'col5' ]; # cols to be accumulated
SET s_tot = {}; SET g_tot = {};
FOREACH i IN sumcols;
SET s_tot.$i = 0; # initialise
SET g_tot.$i = 0;
END;
FOREACH row IN data;
IF s_tot.col2 AND s_tot.col2 <> row.col2; # start of new group
printrow('subtotal', s_tot);
FOREACH i IN sumcols;
SET s_tot.$i = 0; #re-init
END;
END;
printrow('row', row);
SET s_tot.col2 = row.col2; # keep track of group level
FOREACH i IN sumcols;
SET s_tot.$i = s_tot.$i + row.$i;
SET g_tot.$i = g_tot.$i + row.$i;
END;
END;
printrow('grandtotal', g_tot);
-%]
Of course, if you have more than a couple of grouping levels, this can get quite messy. You could make s_tot an array of hashes to manage each level, to avoid hard-coding the levels. That's left as an exercise for the reader, as they say.

MS Access implenting hyperlink like behavior for records to switch between forms

I'm currently working on a Database which requires the following functionality:
For example given a specific project, I have a series of structures which belong to that project, which are displayed in a datasheet view on the project form. I am attempting to allow the user to on double click to navigate to that specific structure which is displayed on another form. Currently I am using filters to implement this behavior, however, this results in the filter being left on, and when I manually turn off the filter, the form I switch to returns back to the first entry.
I am using the current code on the datasheet:
Private Sub struct_name_DblClick(Cancel As Integer)
LookupValue = Me.struct_ID
Form_frm_control.pg_structure.SetFocus
Form_frm_control.subform_structure.Form.Filter = "struct_ID = " & LookupValue
Form_frm_control.subform_structure.Form.FilterOn = True
End Sub
Any help would be greatly appreciated. Thanks in advance.
It all depends on what you need to do.
If you want to display all records and navigate to the selected record, then you can use bookmark navigation:
With Forms!MyOtherForm
.RecordsetClone.FindFirst "struct_ID = " & Me!struct_ID
If Not .RecordsetClone.NoMatch Then
If .Dirty Then
.Dirty = False
End If
.Bookmark = .RecordsetClone.Bookmark
End If
End With
This assumes that the other form is open with all the records loaded.
Another approach to this problem, which I find more useful for popup situations like this, is to just open the other form as a dialog and require it be closed before returning to the calling context. In that case, you'd do this:
DoCmd.OpenForm "MyOtherForm", , , "struct_ID = " & Me!struct_ID, , acDialog
You'd then have to close the other form to get back to the original context.
You might think that with large numbers of records this would be inefficient, but it's actually very fast, as this operation is highly optimized within Access (moreso than filtering an already open form, in fact).