We have Access with Microsoft 365 Apps for business as the frontend to SQL Server 2017.
One of the forms in Access is routinely searched by using filter by form (Ribbon > Home > Sort & Filter > Advanced > Filter by Form). Now performing these steps is taking extraordinarily long and it's not clear to me that anything has changed other than the number of records returned.
Advanced > Clear All Filters
Advanced > Filter by Form
Enter filter criteria into (bound) text boxes, for instance
Toggle Filter
The RecordSource of the form is a SQL statement that's long and contains 9 left joins. It's not pretty, but it has previously worked more quickly.
Unfiltered, the recordset is ~17,500 records. In SSMS the query takes 2 seconds to return. With Filter by Form it's now taking 2+ minutes.
The VM on which SQL Server runs shows performance as normal through Task Manager.
Additional things I've tried:
Rebuilding indexes
Putting the SQL statement into a Query for the form's RecordSource
Creating a basic search form that opens the problematic search form with:
DoCmd.OpenForm "APD_Form", WhereCondition:=sSql
Running the filter by form on someone else's PC
--- none of these have improved performance...
Related
For PowerApps, what data source, other than SharePoint lists are accessible via Powershell?
There are actually two issues that I am dealing with. The first is dynamic updating and the second is the 500 item limit that SharePoint lists are subject to.
I need to dynamically update my data source, which I am currently doing with PowerShell. My data source is not static and updating records by hand is time-consuming and error prone. The driving force behind my question is that the SharePoint list view threshold is 5,000 records however you are limited to 500 visible and searchable records when using SharePoint lists in the Gallery View and my data source contains greater than 500 but less than 1000 records. If you have any items beyond the 500th record that should match the filter criteria, they will not be found. So SharePoint lists are not optional for me until that limitation is remediated
Reference: https://powerapps.microsoft.com/en-us/tutorials/function-filter-lookup/
To your first question, Powershell can be used for almost anything on the Microsoft stack. You could use SQL server, Dynamics 365, SP, Azure, and in the future there will be an SDK for the Common Data Service. There are a lot of connectors, and Powershell can work with a good majority of them.
Take note that working with these data structures through Powershell is independent from Powerapps. Powerapps just takes the data that the data connector gives it, and if you have something updating the data in the background (Powershell, cron job, etc.), In order to get a dynamic list of items, you can use a Timer control and a Refresh function on your data source to update the list every ~5-20 seconds.
To your second question about SharePoint, there is an article that came out around the time you asked this regarding working with large lists. I wouldn't say it completely solves your question, but this article seems to state using the "Filter" function on basic column types would possibly work for you:
...if you’d like to filter the set of items that you are showing in the gallery control, you will make use of a “Filter” expression, rather than the “Search” expression, which is the default that existing apps used. With our changes, SharePoint connector now supports “equals” type of queries on columns that support filtering (Single line of text, choice, numbers, dates and people), so make sure that the columns and the expressions you use are supported and watch for the same warning to avoid reverting back to the top 500 items.
It also notes that if you want to pull from a list larger than the 5k threshold, you would need to use indexes, I have not fully tested this yet but it seems that this could potentially solve your problem.
So I've started learning access due to necessity, as the person who was in charge of it passed way and someone had to keep it going.
I noticed a very bad (at least IMO) behavior in all databases he created: Every single form was bound directly to a table or saved query. This way, if the user opened a form, he had to complete all the steps he was supposed to do, because if he closed the form mid process (or the computer froze, or anything of the sort), the actual data would be compromised as it would be half complete. This often times broke everything in the process chain, rendering sub-sequential steps impossible to be performed and forced me to correct data manually directly in the tables.
As I've start upgrading his stuff and developing my own, I've been trying to learn ways to allow the data to be edited in the form only, making it possible to cancel the process anytime or save the changes all at once in the end.
If the editions were simple, I discovered that I could create a recordset, copy relevant data to unbound fields in the form and, in the end, if the user chose to, copy the data from the form fields back to the recordset.
Other times more complex solutions were required, as I would need to edit several pieces of data at once in continuous forms, "save" them, run more code, maybe add fields to hold the information originated from that processing and so on. I then learned about using temporary tables, but did not like it, since it tended to bloat the db. I even went on to creating temporary databases during code execution that would host the temporary tables and be destroyed in the end, but that added too much unnecessary complexity.
Nowadays I'm using disconnected ADO recordsets to hold the temporary data and fields. It works but has its limitations.
So I'm wondering, what is the best way you - much more experienced than me - guys use to approach this kind of scenario? Is using in memory ADO recordsets really the best way around?
I think you are mixing two things that a form does that have completely different requirements. Editing existing records (and bound forms are great for that) and creating new records (where using a straight bound form can result in creating incomplete records). The way to approach it depends on many things but mainly to how much data is necessary for a new record to be considered "complete".
I usually do one of the following things:
Create an unbound popup modal form for adding new records where only the necessary fields are present. Once complete it loads the new record to the main form for further editing.
Use the above method except the form is not a popup one but a set of unbound fields in the footer or header of the main form.
Let the user create new records but bind validation on the OnClose (and/or other appropriate to your situation) event of the form that deletes the half-filled record if it does not validate.
Let users create new records in the bound form but have a 'cleanup' routine called either on a schedule or based on user actions.
Ultimately if your business process requires the majority of fields to be manually added/edited every time a new record is added or edited, you are better off using an unbound form.
This way, if the user opened a form, he had to complete all the steps
he was supposed to do, because if he closed the form mid process (or
the computer froze, or anything of the sort), the actual data would be
compromised as it would be half complete
No, if the computer freezes, then no data is saved to the table. This is the same if you used a disconnected reocrdset and a un-bound form.
If you use the before update event in the form that has some verification code and does a simple cancel = true, then the forms data is not saved nor is the table updated. Again, if you used a dis-connected record set and the user closes the form, you have to test the data – and again you can either choose to write out the data or not – this effect is ZERO difference from using a bound form to a table or a disconnected form.
If the editions were simple, I discovered that I could create a
recordset, copy relevant data to unbound fields in the form and, in
the end, if the user chose to, copy the data from the form fields back
to the recordset.
No you don’t need to do the above. The above achieves nothing and only racks up additional development hours and increases cost of the application. In near all cases in-bound forms increase development costs over that of a simple form bound to a table. So the original developer had the correct idea. You can control the update of the underlying table in near all cases to achieve the required verification. Forms only save and write the data out if the developer allows as such.
So Access forms when bound no more or less write incomplete data out to a table if you place verification code in the forms before update event. A half-filled bound form, or a half filled un-bound form with dis-connected reocrdset BOTH will not write their data if the computer freezes.
And BOTH types of forms will not write out data to table until such time your verification code has completed.
Access is not designed for un-bound forms, and tools like vb.net, or even VB6 had a whole bunch of cool wizards and support for un-bound forms. In access, we don't have those wizards. And when you use UN-bound forms then you loose tons of form events. You in effect get the worst of both worlds, since you lose use of form events and have no wizards or support for un-bound. Even just the several delete record events we have are rather amazing.
You lose use of me.dirty, on-insert, me.newReocrd, forms after update events - the list of features you toss out and lose is huge. And if you want a button to write data to the table (such as a save button on the form), then just go:
If me.Dirty = True then
me.Dirty = False ' this forces your verification code to run
End if
There are FEW use cases in which in-bound forms will benefit you, but they will cost you rather much in terms of development times.
I'm working on the design of a database for a client and they would like to work with one of their companies (customers) at a time throughout the entire session.
Ideally I would like to set up the forms so that they select the company to work with first and then can navigate to various other forms to manage the different aspects of that company (locations, departments, employees, etc.). There will obviously be many companies in the database, so I would need to ensure that they only see records for the selected company and adding new values ties them to the selected company.
I've been doing some searching around for how best to accomplish this but I've had no real luck.
Any guidance you can give would be awesome.
PS. I've got the structure of the database tables setup to ensure that any entity that is company-specific has a foreign key that identifies the company.
When the user enters the program, the first form they should see should be the company chooser form. Then the user can continue on to the other forms, which would all be filtered based on the user's initial selection.
You can either have the form write to a module-level variable, and all the forms would be filtered by the value of a function which would return the company ID to filter by.
It may be easier for you to simply hide the form, and filter all the successive forms by the value in the combo box of the hidden form.
You might also want to consider creating a query for each table, which contains only the filtered values per company (either with a function, or by the value of the hidden form) and basing all your forms/reports on those queries. That way you won't accidentally forget to include the filter in a form/report.
I'm using Access 2010 and I would like to create a form where useres can enter data much like they can in an Excel format. The users need to enter a date when a step is complete for a specific unit.
In Excel the units run aross the top row, the steps run down the left column. In the cell where the unit and step meet, you put a date when it is done. In Access it is much harder to create a form that looks like this (at least it has been for me so far). I tried to use a crosstab query, but you cannot enter information into a crosstab query. I can do a massive form listing every combination of units and steps, but there are over 50 units and each one has 63 steps. I don't want users to have to look through a form with over 3,000 lines in it just to enter a date completed.
This definitely feels like something we should be using a database for instead of an Excel spreadsheet. I have the format for the back end tables to hold the data. My problem is how to create a form that is easy for the user to enter the dates.
I think you have three options. First, you could build the form from scratch, which you've already admitted would suck. And it would take a lot of VBA to get the data in the right place. Second, you could automate Excel - it's the same as the first method but you don't have to build the form. You would populate an Excel spreadsheet and write the changes back to the database. This is not a trivial task and I don't recommend it.
The third option is to throw away the idea that you want to do this in a grid. You've probably been doing it in Excel and to the extent that you replicate that, your users will be happy and comfortable. Moving to a database gives you benefits, but there are costs. This is one of the costs.
So create a form with a dropdown containing all of the units. When a unit is selected, populate a subform (datasheet view) listing all of the procedures and dates. The user can then select the unit, and fill in the date next to whichever procedures he wants. Then he can select another unit and do the same thing. If you set it up right, your data will flow to the tables automatically and you'll get all the benefits of data validation, input masks, relationships, and whatever else you're using at the table level.
I know that's not what you were looking for, but I think it's the best way forward.
I have created a complex sql server 2008/coldfusion search page, that searches thru a variety of tables.
On the left is a list of the categories, plus an everything category, by each category or type of result is a total number of results of that type found in the current search result.
I have everything fine, but I am hoping there is a more optimal approach.
Because everytime i filter the search to a specific category, i still have to get all the results, so as to make sure the everything category has the correct totals.
And because of this, I have realized this is a problem I've had in lots of other programs in coldfusion/sql.
Where you want to reduce the number of results by some field in the select, but you need to keep the original recordcount total.
But you really don't want to re-run the whole massive query everytime, when you just need to get the trimmed results.
This program is 1 cfc, 1 cfm, 1 stored procedure, and jquery/ajax inside the cfm to call the cfc.
The cfm calls the cfc when it originally get's a form submitted search request, and then any filtering does the same thing.
However if there are more than 20 results then it show's a button at the bottom to do via ajax get 20 more records.
My main goal is to improve performance, make sure i keep an accurate record of what the record count is before any filtering is done, without having to rerun the unfiltered query every time.
This is a kind of complex problem, so there might not be any answers...
Thank you all for trying..
I would run the "big" query once, then pop it into a SESSION variable. Then I'd use Query-of-Query to return subsets based on filters.
The main query always exists, so you can query against that or use metadata like bigQuery.recordCount. Your QofQ is a smaller set of data you can use for display. And you can re-apply filters without having to return to the database.
Well you need to run the query (or a count(*)) at least once to get the total number. You could:
Cache this query and refer to the
cached query's recordcount again
and again
Store the record count in the session scope until the next time it is run for this user