I have hundreds of rows of raw data in a CSV. It is stacked like this:
ID.....SS.....P.....Window
S1235...345...48.....Fall
S1235...460....68....Winter
S1123....389....50....Fall
S1123.....598....98.....Winter
What I would like to do is match it up so that all of the IDs match with similar ID on one row only (without doing a manual copy and paste) so it looks like this:
ID.....SS.....P.....Window
S1235...345...48.....Fall..... S1235...460....68....Winter
S1123....389....50....Fall.....S1123.....598....98.....Winter
I don't care if there are multiple columns...because I will manually delete what I don't need. What I need is for all the data with the same ID to appear on one row only. Each piece of information needs to be in it's own cell.
I appreciate the help in advance...I'm not very good with this stuff...but it would be a HUGE time saver if you can help me out!
Related
In my model I want to loop through the database which contains multiple columns (see example) by an event. The idea behind it is that I want to create dynamic events based on the rows in the database.
But I've no clue how to iterate through a database in anylogic and also was not able to find an example of a loop with a database.
The dummycode of my problem would look something like this:
For order in orderdatabase:
Create order based on (order.name, order.quantity, order.arrivaltime, order.deliverylocation)
Where order in the loop is every row of the database, and the value on which the creation is based based on the different column values of that specific row.
Can somebody give me a simple example of how to create such a loop for this specific problem.
Thanks in advance.
Use the database query wizard:
put your cursor into a code field
this will allow you to open the database wizard
select what you need (in your case, you want the "iterate over returned rows and do something" option
Click ok
adjust the dummy code to make it do what you want
For details and examples, check the example models and the AnyLogic help, explaining all options in detail.
I am trying to insert data to big query using google cloud dataprep, I did create recipe and add first row as header row, but when I am trying to run on multiple files it insert the header row to my big query table also.
Anybody facing this problem ?
Welcome to StackOverflow, Andy!
I think I'm correctly understanding your problem, but I want to make sure since I'm making some assumptions:
You have multiple files imported in Dataprep
You created a recipe for the first file and convert row 1 to be a header
You apply a UNION step to merge the additional files
Your output contains the header rows for the additional files
If that's correct, the issue is that the header rows in the other files aren't being removed simply because Dataprep doesn't know what they are. In most cases, Dataprep will detect the file structure and you won't have to manually specify the header row. When that fails, however, UNION steps get a little funny like this—but you can definitely fix it in Dataprep.
Workarounds:
Apply a Recipe to Each Input File
Simply add a recipe to each file that converts the first row to a header—then instead of selecting your original file in the main recipe's UNION, select the other recipes (Dataprep will run them before merging the data).
While this takes some extra effort, it's doable for a small number of files. The advantage here is that you don't have to worry about whether your data may contain the header value—but I'd recommend using the other option if you're able to.
Use a Custom Filter Formula to Delete All Header Rows
The other option is a bit more dependent on your data, but lets you do everything in the main recipe. For example, after setting headers from the first file and applying your UNION you would add a "Filter rows using custom formula" step (or clicking Filter Rows > On column values > Custom Filter...), then match using a column that wouldn't contain the header string (e.g. CustomerID == "CustomerID")—integer columns work great since you don't have to worry if the value could contain the header string. The Resulting wrangle script should look something like this:
header sourcerownumber: 1
[union step goes here]
filter type: custom rowType: single row: CustomerID == 'CustomerID' action: Delete
Note: You may be tempted to do this by using $sourcerownumber, but that doesn't exist due to the union. I'm hoping that they'll eventually support it for this use case though.
These aren't the only ways you could eliminate the headers, but should provide two easy options for you.
As a pro-tip, you can copy a line of the wrangle script above and paste it after clicking "New Step" in your recipe and it'll set up the filter the same way that I did so you don't have to start from scratch. Just change the column name/value and you should be good to go.
Again, welcome to the site—and if any of the assumptions above are incorrect, update your original question with the additional details and let me know in a comment and I'll be happy help you out further.
My company uses a third-party vendor to get all of our NPS information. I'm trying to set up a data feed from this vendor into our data warehouse, which runs PostgreSQL.
The feed is in the form of 2 tab-separated text files: "question mapping" and the responses. The question map is one row per question, with columns for question id, question text, question label question type, etc - straightforward. The responses are one row per survey response, with a column for each question and stuff like user id, etc. Here are the 2 biggest problems:
The survey questions sometimes use the same question ID for different questions, resulting in multiple columns in the response data having the same name but not being the same question.
The number of questions could change, resulting in a different number of columns in the data.
Both of these things make it a real headache to automate a data feed into a single table.
I'm afraid I don't quite know how to phrase my real question other than, "Does anyone have any ideas how I can accomplish this?" If I think of something better than that, I'll come and update this, so for now:
Does anyone have any ideas at all about how I can efficiently set up my automated data feed without having to always drop and recreate everything?
If your data is a mess and doesn't really have well defined columns you can use the entity attribute value pattern, where you turn each fact into a set of rows with 4 columns - a unique row id, the same entity id for each row extracted from the map, an attribute column (where you put what would be the name of the column) you get from the key of the map, and a value column where you put the value from the map. It's not that neat but you can still query it and you won't have to drop it when you receive a map with a new column.
I'm operating in Word 2013 and 2010, so I can use code that works in either. I'm trying to create a word document to keep track of my recipes. At its most basic I want to have a TOC that updates based on headings. I also want it to have any category I want (eg: Appetizers, Drinks, Entrée, etc...) ordered alphabetically. Under each category I have tables. Each recipe gets a table that has it's name, directions, notes, tags, and potentially a picture. The second cell has another two column table inside of it that contains the quantity and name of each ingredient necessary for the recipe.
I have all of that so far and I'd like to automate adding new categories and recipes. Currently, I have to find the category, then scroll down to find where the name goes alphabetically and insert a quick table I made. I then fill in the info.
I'd like to be able to search the document for each category name, then insert the new category wherever it belongs, with a space before and after it. I found that my tables give me trouble if I don't have a space between everything. It tries to pull anything it's touching into the table and merge them.
I wanted to give the backstory, so you'd know where I was going to go eventually and could provide help that fits better with what I need. After I can add a new category, I plan to use vba to organize each table alphabetically by the name in the first cell of each table. It will also help when I start adding sorts to it. Eventually, I'd like to be able to sort it to say, only display recipes from a certain person, or display my frequently used recipes. I'd then have it either hide all the others or create a new doc with just these. So thanks for the help. Below I'll post the code I most recently tried. I tried a few other variations of this same code and keep getting an 'expected end of statement. I've gotten other errors when trying other variations of it, but this is the best I can come up with on my own.
Private Sub UserForm_Initialize()
For Each cat In ActiveDocument.Styles = "Heading 1"
lstCat.AddItem (cat)
Next
End Sub
I have a form called frmAddCategory I'm using as a test. I was going to have a listbox lstCat to show every category with the style heading 1. I have a textbox called txtAdd to type new ones and a cmdAdd button to add it to the form.
Edit: I've been playing around with my macro recorder after finding out about outline view mode. I set it to show only 'heading 1' level and selected the ones I wanted, not selecting the appendix or reference. Then I went to the home tab and sorted paragraph by ascending alphabetical order. I got some code I believe I can use to get it to run in VBA. However, it's not a complete fix as I don't want to select the last two with heading 1. It also works if I manually select the tables under each heading 1, but I can't set the spacing before and after. I'd like each heading and the tables under them to have a space or two between each for looks and editing purposes.
Also, if someone is going to give my question a negative rating, then please post a comment explaining. As far as I can tell from the faq about the forum and the other questions I've seen, it is a well posed question. A clear title, a good explanation of the problem, code examples, research. So if I am doing something wrong, please inform me, so that I can correct it.
first thanks for your bit about macro recorder and outline mode, I have been trying for long time to fill a list box with selection.text between two HeadingLevel(1) headings.
now to yours, sorry I can't think of way to do in word. BUT it would be real easy in Access. one table of categories like called tblCategories another for recipes tblRecipes. To make real easy, when the use autoID on ALL tables. But to avoid LOTS of headaches for tblRecipes rename its autoID to RecipesID same for other tables. in table of recipes you can use a memo field to hold large amount of data. the spot for text in Heading would be put in one field of the tblRecipes. once you have tables looking to have a field for each item you want to track. hit save, then use wizard to create a form based on table. repeat for all tables you want to have it real easy to put info into any table.
1.reportTOC based on query of Every Heading you want, can preview or print as want. reportByCategory and so on reports are sorted a to z unless you want to sort by Owner, then recipe all auto sorted a - z.
report wizard to get hard copy. if you want to sort real easy, built in. also if want to be able to pick all recipes for a holiday real easy, one table tblHolidays. one tblHolidayRecipes fields autoId (not used by you anywhere but needed), fldRecipesId (holds RecipesId) , fldHolidayId (holds HolidayId). the wizard will show how to get only what you want. in access 2013 you can include pix of food or.
I'm trying to copy some field values to a duplicate database. One record at a time. This is used for history and so I can delete some records in the original database to keep it fast.
I don't want to manually save the values in a variable because there are hundreds of fields. So I want to go to the first field, save the field name and value and then go over to the other database and save the data. Then run a 'Go to Next Field' and loop through all the fields.
This works perfectly, but here is the problem: When a field is a calculation you cannot tab into it and therefore 'Go to Next Field' doesn't work. It skips it.
I though of doing a 'Go to Object' but then I need to name all the objects and I can't find a script to name objects.
Can anyone out there think of a solution?
Thanks!
This is one of those problems where I always found it easier to do an export/import.
Export all the data you want from the one database, and then import it into the other database. All you need to do is:
Manually specify which fields you want to copy
Map the data from the export to the right fields in the new database/table
You can even write a script to do these things for you.
There are several ways to achieve this.
To make a "history file", I have found there are several cases out there, so lets take a look.
CASE ONE
Single file I just want to "keep" a very large file with historical data, because I need to erease all data in my Main file.
In this case, you should create a "clone" table (in the same file ore in other file, is the same). Then change any calculation field to the type of the calculation result (number, text, date, an so on...). Remove any "auto entered value or calculation from any field, like auto number, auto creation date, etc..). You will have a "Plain Table" with no calculations or auto entered data.
Then add a field to control duplicate data. If you have lets say an invoice number (unique) for each record, you can do this to achieve this task. But if you do not have a unique field that identifies the record as unique, then you have to create one...
To create such a field, I recommed to add a new field on the clone table and set as an aunto entered calculation and make a field combination that is unique... somthing like this: invoiceNumber & "-" & lineNumber & "-" " & date.
On the clone table make shure that validation is set up for "always", and no empty values allowed and that this value is unique.
Once you setup the clone table... then you can import your records, making sure that the auto enty option is on. Yo can do it as many times as you like, new records will be added and no duplicates.
If you want, can make a Script to do the move to historical table all the current records before deleting them.
NOTE:
This technique works fine when the data you try to keep do not have changes over time. This means, once the record is created is has no changes.
CASE TWO
A historical table must be created but some fields are updated.
In the beginnig I thougth a historical data, never changes. In some cases I found this is not the case, like the case I want to track historical invoices but at the same time, keep track if they are paid or not...
In this case you may use the same technique above, but instead of importing data... you must update data based on the "unique" fields that identifiy the record.
Hope this technique helps
FileMaker's FieldNames() function, along with GetField() can give you a list of field names and then their values