!!! UPDATED !!!
We have spreadsheets of complex product data coming in from multiple sources (internal, customers, vendors).
Since the authorship is so diverse, it's impractical to try governing formatting details such as column order and the number of header-rows.
These CSV spreadsheets will be uploaded to our DB via an existing form.
(My first Zend_Form ... I'm almost done with it)
The user needs to see a sample from a given spreadsheet so they can Map the columns and start-row.
To achieve that, I need to generate an html table of that dynamic content, and weave the form elements in and around the table data.
The user would select which values are to be found in each column, and identify the first row of data (after any header rows).
CLICK HERE to see an example.
(NOTE: Most of my work here is under an NDA, so contrived examples is the best we can get :)
In this example, I'd expect the output to be:
_POST('first_row'=>2, 'column0'=>'mi', 'column1'=>'lName', 'column2'=>'fName', 'column3'=>'gender')
With all those scpecifics mapped/defined, the uploaded spreadsheet can then be parsed and accurate data can be added to the product_history database.
Is ZF a good tool for this particular problem, or should I just write something from scratch?
How would you aproach this?
I am finally JUST BARELY starting to get this ZF stuff straight in my head, and this one has got me totally lost :)
Any and All advice appreciated.
~ Mo
I think in your case, using Zend_Form would be helpful for this situation.
The tricky part to it is of course that your forms are going to be largely dynamically generated on-the-fly based on the header and first row content of the CSV file.
Whether you used Zend_Form, or pure PHP, or some other solution, a lot of what you will be doing is the same (analyzing the CSV, providing dynamic inputs based on the CSV, and then error checking the selections). I think using Zend_Form has the advantage of making something like this very cleanly.
Given Zend_Form's nature, e.g. how it validates existing forms based on the elements added to the Zend_Form itself, you need to take a special approach with the form. Basically, after the user uploads the CSV once, you will create a Zend_Form object based on the number of columns, their positions in the CSV, and the name of the column.
Since you don't want to bother the user to upload the CSV multiple times if they make incorrect selections, I would parse the CSV into some sort of structure, maybe a simple object or array, and then build your Zend_Form based on that data. This way, you can save that structure to the session, so you can continue to regenerate the form based on the parsed data without having to read the file each time. This is because the main challenge with Zend_Form and dynamic forms, is that not only does the form need all of the elements and their properties when you want to display the form, but they are also required in order to validate the form and re-display the validated form.
I remember seeing this functionality many years ago in a PHP script, which I found is still available. Perhaps you could look at it for ideas. I won't post the link here since the screenshots and script are mostly adult website related and the site is NSFW for the most part, but it is called TGPX by JMBSoft. The 7th of the 8th screenshot on the main product page shows the import process where it lets the user map fields to data, exactly what you are doing.
Hope my advice is helpful, feel free to comment with any questions.
Related
We regularly need to perform a handful of relatively simple tests against a bunch of MS Word documents. As these checks are currently done manually, I am striving for a way to automate this. For example:
Check if every page actually has a page number and verify that it is correct.
Verify that a version identifier in the page header is identical across all pages.
Check if the document has a table of contents.
Check if the document has a table of figures.
Check if every figure has a caption.
et cetera. Is this reasonably feasible using PowerShell in conjunction with a Word API?
Powershell can access Word via its object model/Interop (on Windows, at any rate) and AIUI can also work with the Office Open XML OOXML) API, so really you should be able to write any checks you want on the document content. What is slightly less obvious is how you verify that the document content will result in a particular "printed appearance". I'm going to start with some comments on the details first.
Just bear in mind that in the following notes I'm just pointing out a few things that you might have to deal with. If you're examining documents produced by an organisation where people are already broadly speaking following the same standards, it may be easier.
Of the 5 examples you give, without checking the details I couldn't say exactly how you would do them, and there could be difficulties with all of them, but for example
Check if every page actually has a page number and verify that it is correct.
Difficult using either OOXML or the object model, because what you would really be checking is that the header for a particular section had a visible { PAGE } field code. Because that field code might be nested inside other fields that say "if don't display this field code", it's not so easy to be sure that there would be a page number.
Which is what I mean by checking the document's "printed appearance" - if, for example, you can use the object model to print to PDF and have some mechanism that lets PS inspect the PDF's content, that might be a better approach.
Verify that a version identifier in the page header is identical across all pages.
Similar problem to the above, IMO. It depends partly on how the version identifier might be inserted. Is it just a piece of text? Could it be constructed from a number of fields? Might it reference Document Properties or Variables, or Custom XML content?
Check if the document has a table of contents.
Perhaps enough to look for a TOC field that does not have certain options, such as a \c option that a Table of Figures would contain.
Check if the document has a table of figures.
Perhaps enough to check for a TOC field that does have a \c option, perhaps with a specific parameter such as "Figure"
Check if every figure has a caption.
Not sure that you can tell whether a particular image is "a Figure". But if you mean "verify that every graphic object has a caption", you could probably iterate through the inline and floating graphics in the document and verify that there was something that looked like a Word standard caption paragraph within a certain distance of that object. Word has two standard field code patterns for captions AFAIK (one where the chapter number is included and one where it isn't), so you could look for those. You could measure a distance between the image and the caption by ensuring that they were no more than a predefined number of paragraphs apart, or in the case of a floating image, perhaps that the paragraph anchoring the image was no more than so many paragraphs away from the caption.
A couple of more general problems that you might have to deal with:
- just because a document contains a certain feature, such as a ToC field, does not mean that it is visible. A TOC field might have been formatted as not visible. Even harder to detect, it could have been formatted as colored white.
- change tracking. You might have to use the Word object model to "accept changes" before checking whether any given feature is actually there or not. Unless you can find existing code that would help you do that using the OOXML representation of the document, that's probably a strong case for doing checks via the object model.
Some final observations
for future checks, perhaps worth noting that in principle you could create a "DocumentInspector" that users could call from Word BackStage to perform checks on a document. Not sure you can force users to run it, or that you could create it in PS, but perhaps a useful tool.
longer term, if you are doing a very large number of checks, perhaps worth considering whether you could train a ML model to try to detect problems.
Recently within our team, we were discussing to create a generic form creation tool which can create dynamic form with desired fields (text box, drop down, radio button etc. etc.)
the user can add as many as fields he wants to add (though normally it would not exceeds more than 20), can share the form with other users who can answer the questions or it could be simply just a data entry form for the single user.
We are trying to execute this step by step and in the first phase we would be doing something like:-
the user will have the option to create a new form.
there would be few predefined fields exists i.e. name label and textbox, address label and textarea, state drop down with some values, gender radio with options and some other sample fields.
above-displayed fields on the newly created form are just to help user to understand what kind of different fields he can create.
user can add/ delete fields, fields type would be textbox, textarea, dropdown, radio, multi-select checkbox etc.
after completing the desired fields user would save the form with "create form" button. entire data -field type, their values if it is drop down radio, will get saved in the database.
To access the form user will access the URL like form/5 and fill it and save.
We are creating a dynamic form and saving the form inputs as well.
My Initial Thoughts
When user will create a new form entire data would be saved as a single JSON object on NoSQL database.
when other or same user try to access the created from to fill the values same JSON object will be fetched and HTML would be created, In phase 2 we would might need a JSON to HTML converter program. but to keep it simple for now, we would just generate HTML from JSON at the client side. (web app/mobile app)
after users complete their inputs and save it. a new JSON object will be saved in another table named Records.
I read somewhere that for reporting and analytics purpose relational database is recommended so I was thinking of Maintaining all data in another relational database as well, duplicating our entire database. I am not sure about this and it could be a bad design.
This application can replace our several applications - some survey application, few data forms thus no. of records could easily reach into millions.
I am looking forward to any suggestion on the architecture/design and database type.
https://www.makeuseof.com/tag/finest-14-google-forms-alternatives-try/
Update 1: I agree with #SerejaBogolubov comment and not looking for the High-level/low-level design that includes Classes, Interfaces, their relations, functions and design patterns.
The question mentioned few steps and building block I would be using at the start and would like to know about various recommendations and thoughts.
Honestly, I see no design ideas in your post. The place you gonna save (and format of the text file or whatever else) is secondary. Design is about interfaces, functions signatures, applied patterns, etc.
As a suggestion, you (obviously) have to separate form representation from the way it is rendered. Thus something like FormRepresentation -> Html appears. Then the question is how you gonna structure your FormRepresentation? I would suggest a (monoidic) binary operation: FormItem -> FormItem -> FormItem also known as kind of Composite OOP pattern; within associativity of the composition you might get parallezation for free and avoid performance headache, as well as keep your signatures really simple yet meaningful.
So, that was just a sketch. I would offer you to update your post appropriately such that design details (rather than implementation details) become clear.
I am working on Access 2007. I have a table with some fields in it. I had created a form from the table and one of the fields of the table is a concatenation of 2 fields from the same table.
There are 2 fields OppNo and Material in the table. I had created a form with these (and others in the table) fields. There is another field OppMat which is blank in the table. However, I had got the data into OppMat field populated as a concatenation of OppNo and Material fields in the form. I am now looking at having the table updated with the data of OppMat from the form to the same corresponding field in the table.
Kindly advise as how I could achieve this.
Thanks and regards,
This might be possible if these cases are met:
The field OppMat ALWAYS has the same structure and you can assure that it does.
Users will not be able or very unlikely to deviate from this structure.
Notably you should use something like Left(), Right(), Mid() and so on. Whatever works best for your structure. You can use the string modifiers: https://msdn.microsoft.com/en-us/library/dd789093.aspx
On the other hand I must admit that I am sceptical if your solutions is the best. Here are my reasons:
Users are unpredictable and will continously deviate from your intended way to use the application.
Using the string functions and fixating on one structure makes your code construction inflexible, hard to maintain if changes occur and prone to errors.
The alternatives seem to be better with little drawback.
My suggestions for alternatives:
Use one textbox for each field so that you have two textboxes. You can even position them that way that it almost looks like it is continous text. But not too much otherwise the user will beconfused.
You can add a label that shows your concatenated fields. But for input you use two different textboxes.
Cheers!
I have a situation where we have a base recordset with about one hundred thousand records. And, we are creating a separate application that shares some of the dataset, but not most, so we are creating a detail table that has a one to one relationship with the original table. What I want to do is pull the existing information from the original table and display it as read only, but I want the fields in the detail table to be writeable.
I've started by creating a listing with the contents of the original table, I want the user to be able to seemlessly hit "edit" by an entry and be taken to the form to create the detail record, click save, and have it update, or create the record.
So, what is the best way to do this?
This question is old, but in order to help future searchers...
Including a related table in a form is handled automatically if your DBIx::Class result sources are correctly setup. A related table can be pulled into the form by simply defining a compound field using the relationship name of the other field and defining subfields with the names of the columns in the related table.
See: HTML::FormHandler::Manual::Fields
And: HTML::FormHandler::Manual::Database
This isn't a real answer, in that I can't really provide you with a solution, but I can hopefully provide you with somewhere to start. I think you need to define a form in HTML::FormHandler which represents a single row from your detail table, then build your read-only form from multiple instances of single-row form. This article describes the problem and a partial solution much better than I have:
http://catdev.blogspot.com/2009/05/defining-form-processing-problem.html
I'm fairly new to Perl and HTML::FormHandler, and there's a good chance that there is a better way of doing this. It's a problem that I have solved 'manually' in PHP before, but one which I am sure I will come across in Perl.
What are some possible designs to deal with frequently changing data forms?
I have a basic CRUD web application where the main data entry form changes yearly. So each record should be tied to a specific version of the form. This requirement is kind of new, so the existing application was not built with this in mind.
I'm looking for different ways of handling this, hoping to avoid future technical debt. Here are some options I've come up with:
Create a new object, UI and set of tables for each version. This is obviously the most naive approach.
Keep adding all the fields to the same object and DB tables, but show/hide them based on the form version. This will become a mess after a few changes.
Build form definitions, then dynamically build the UI and store the data as some dictionary like format (e.g. JSON/XML or maybe an document oriented database) I think this is going to be too complex for the scope of this app, especially for the UI.
What other possibilities are there? Does anyone have experience doing this? I'm looking for some design patterns to help deal with the complexity.
First, I will speak to your solutions above and then I will give my answer.
Creating a new table for each
version is going to require new
programming every year since you will
not be able to dynamically join to
the new table and include the new
columns easily. That seems pretty obvious and really makes this a bad choice.
The issues you mentioned with adding
the columns to the same form are
correct. Also, whatever database you
are using has a max on how many
columns it can handle and how many
bytes it can have in a row. That could become another concern.
The third option I think is the
closest to what you want. I would
not store the new column data in a
JSON/XML unless it is for duplication
to increase speed. I think this is
your best option
The only option you didn't mention
was storing all of the data in 1
database field and using XML to
parse. This option would make it
tough to query and write reports
against.
If I had to do this:
The first table would have the
columns ID (seeded), Name,
InputType, CreateDate,
ExpirationDate, and CssClass. I
would call it tbInputs.
The second table would have the have
5 columns, ID, Input_ID (with FK to
tbInputs.ID), Entry_ID (with FK to
the main/original table) value, and
CreateDate. The FK to the
main/original table would allow you
to find what items were attached to
what form entry. I would call this
table tbInputValues.
If you don't
plan on having that base table then
I would use a simply table that tracks the creation date, creator ID,
and the form_id.
Once you have those you will just need to create a dynamic form that pulls back all of the inputs that are currently active and display them. I would put all of the dynamic controls inside of some kind of container like a <div> since it will allow you to loop through them without knowing the name of every element. Then insert into tbInputValues the ID of the input and its value.
Create a form to add or remove an
input. This would mean you would
not have much if any maintenance
work to do each year.
I think this solution may not seem like the most eloquent but if executed correctly I do think it is your most flexible solution that requires the least amount of technical debt.
I think the third approach (XML) is the most flexible. A simple XML structure is generated very fast and can be easily versioned and validated against an XSD.
You'd have a table holding the XML in one column and the year/version this xml applies to.
Generating UI code based on the schema is basically a bad idea. If you do not require extensive validation, you can opt for a simple editable table.
If you need a custom form every year, I'd look at it as kind of a job guarantee :-) It's important to make the versioning mechanism and extension transparent and explicit though.
For this particular app, we decided to deal with the problem as if there was one form that continuously grows. Due to the nature of the form this seemed more natural than more explicit separation. We will have a mapping of year->field for parts of the application that do need to know which data is for which year.
For the UI, we will be creating a new page for each year's form. Dynamic form creation is far too complex in this situation.