I am following the example: https://github.com/Shopify/dashing/issues/78#issuecomment-14940695
The eventmachine.watch_file returns "file_deleted" everytime I modify the spreadsheet. It should actually return "file_modified"
Can someone help me?
require 'roo'
EM.kqueue = EM.kqueue?
file_path = "#{Dir.pwd}/spreadsheet.xls"
def fetch_spreadsheet_data(path)
s = Roo::Excel.new(path)
send_event('career_score', { current: s.cell(2, 2) })
end
module Handler
def file_modified
puts "BLAHBLAH1"
fetch_spreadsheet_data(path)
puts "BLAHBLAH"
end
def file_moved
puts "Moved"
fetch_spreadsheet_data(path)
end
def file_deleted
puts "Deleted"
fetch_spreadsheet_data(path)
end
def unbind
puts "#{path} monitoring ceased"
end
end
fetch_spreadsheet_data(file_path)
EM.next_tick do
EM.watch_file(file_path, Handler)
end
Seems that roo has problem with XLS files. Works fine after I convert it to CSV.
Related
I have a program which exports a System.Data.DataTable to an XLSX / OpenXml Spreadsheet. Finally have it mostly working. However when opening the Spreadsheet in Excel, Excel complains about the file being invalid, and needing repair, giving this message...
We found a problem with some content in . Do you want us to
try to recover as much as we can? If you trust the source of the
workbook, clik Yes.
If I click Yes, it comes back with this message...
Clicking the log file and opening that, just shows this...
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<recoveryLog xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
<logFileName>error268360_01.xml</logFileName>
<summary>Errors were detected in file 'C:\Users\aabdi\AppData\Local\Temp\data.20190814.152538.xlsx'</summary>
<repairedRecords>
<repairedRecord>Repaired Records: Cell information from /xl/worksheets/sheet1.xml part</repairedRecord>
</repairedRecords>
</recoveryLog>
Obviously, we don't want to deploy this into a production environment like this. So I've been trying to figure out how to fix this. I threw together a quick little sample to validate the XML and show the errors, based on this link from MSDN. But when I run the program and load the exact same XLSX document that Excel complains about, the Validator comes back saying that the file is perfectly Valid. So I'm not sure where else to go from there.
Any better tools for trying to validate my XLSX XML? Following is the complete code I'm using to generate the XLSX file. (Yes, it's in VB.NET, it's a legacy app.)
If I comment out the line in the For Each dr As DataRow loop, then the XLSX file opens fine in Excel, (just without any data). So it's something with the individual cells, but I'm not really DOING much with them. Setting a value and data type, and that's it.
I also tried replacing the For Each loop in ConstructDataRow with the following, but it still outputs the same "bad" XML...
rv.Append(
(From dc In dr.Table.Columns
Select ConstructCell(
NVL(dr(dc.Ordinal), String.Empty),
MapSystemTypeToCellType(dc.DataType)
)
).ToArray()
)
Also tried replacing the call to Append with AppendChild for each cell too, but that didn't help either.
The zipped up XLSX file (erroring, with dummy data) is available here:
https://drive.google.com/open?id=1KVVWEqH7VHMxwbRA-Pn807SXHZ32oJWR
Full DataTable to Excel XLSX Code
#Region " ToExcel "
<Extension>
Public Function ToExcel(ByVal target As DataTable) As Attachment
Dim filename = Path.GetTempFileName()
Using doc As SpreadsheetDocument = SpreadsheetDocument.Create(filename, DocumentFormat.OpenXml.SpreadsheetDocumentType.Workbook)
Dim data = New SheetData()
Dim wbp = doc.AddWorkbookPart()
wbp.Workbook = New Workbook()
Dim wsp = wbp.AddNewPart(Of WorksheetPart)()
wsp.Worksheet = New Worksheet(data)
Dim sheets = wbp.Workbook.AppendChild(New Sheets())
Dim sheet = New Sheet() With {.Id = wbp.GetIdOfPart(wsp), .SheetId = 1, .Name = "Data"}
sheets.Append(sheet)
data.AppendChild(ConstructHeaderRow(target))
For Each dr As DataRow In target.Rows
data.AppendChild(ConstructDataRow(dr)) '// THIS LINE YIELDS THE BAD PARTS
Next
wbp.Workbook.Save()
End Using
Dim attachmentname As String = Path.Combine(Path.GetDirectoryName(filename), $"data.{Now.ToString("yyyyMMdd.HHmmss")}.xlsx")
File.Move(filename, attachmentname)
Return New Attachment(attachmentname, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")
End Function
Private Function ConstructHeaderRow(dt As DataTable) As Row
Dim rv = New Row()
For Each dc As DataColumn In dt.Columns
rv.Append(ConstructCell(dc.ColumnName, CellValues.String))
Next
Return rv
End Function
Private Function ConstructDataRow(dr As DataRow) As Row
Dim rv = New Row()
For Each dc As DataColumn In dr.Table.Columns
rv.Append(ConstructCell(NVL(dr(dc.Ordinal), String.Empty), MapSystemTypeToCellType(dc.DataType)))
Next
Return rv
End Function
Private Function ConstructCell(value As String, datatype As CellValues) As Cell
Return New Cell() With {
.CellValue = New CellValue(value),
.DataType = datatype
}
End Function
Private Function MapSystemTypeToCellType(t As System.Type) As CellValues
Dim rv As CellValues
Select Case True
Case t Is GetType(String)
rv = CellValues.String
Case t Is GetType(Date)
rv = CellValues.Date
Case t Is GetType(Boolean)
rv = CellValues.Boolean
Case IsNumericType(t)
rv = CellValues.Number
Case Else
rv = CellValues.String
End Select
Return rv
End Function
#End Region
For anyone else coming in and finding this, I finally tracked this down to the Cell.DataType
Setting a value of CellValues.Date will cause Excel to want to "fix" the document.
(apparently for dates, the DataType should be NULL, and Date was only used in Office 2010).
Also, if you specify a DataType of CellValues.Boolean, then the CellValue needs to be either 0 or 1. "true" / "false" will also cause Excel to want to "fix" your spreadsheet.
Also, Microsoft has a better validator tool already built for download here:
https://www.microsoft.com/en-us/download/details.aspx?id=30425
So I have some code which uses xlwings for writing data in Excel file, xlsm.
after i've done writing, I press a certain button to calculate.
sometimes, an error/message pops in the Excel, which is fine, but i want to catch this message to python, and write it later to a log/print it.
also, i need to interact with this message, in this case to press "Ok" in the message box
Attached image of the message box
So guys, I've been able to solve this with an external python library.
here is the code:
from pywinauto import application as autoWin
app = autoWin.Application()
con = app.connect(title = 'Configuration Error')
msgText = con.Dialog.Static2.texts()[0]
con.Dialog.Button.click()
con.Dialog.Button.click()
print(msgText)
basically, what it does, is connecting to the app, and searching for the title.
in this case "Configuration Error"
it needs to perform double click in order to press "Ok" to close the message.
Secondly, it gets the text from the message, and can forward it wherever i want.
important part to remember though, because this should be an automated task, it should run concurrently, which means Threading.
so, a simple Thread class below:
class ButtonClicker(Thread):
def __init__(self):
Thread.__init__(self)
self._stop_event = Event()
def stop(self):
self._stop_event.set()
def stopped(self):
return self._stop_event.is_set()
def run(self) -> None:
while True:
time.sleep(3)
try:
app = autoWin.Application()
con = app.connect(title='Configuration Error')
msg_data = con.Dialog.Static2.texts()[0]
while True:
con.Dialog.Button.click()
# con.Dialog.Button.click()
# print(msg_data)
return msg_data
except Exception as e:
print('Excel didnt stuck')
break
and of course to actually use it:
event_handle = ButtonClicker()
event_handle.start()
some manipulation is needed in order to work in different codes/scenarios, but at least I hope i will help others in the future, because this seems to be very common question.
#Danny's solution, i.e. pywinauto and Thread, works perfectly in my local machine, but it seems can't catch the message box when Excel is running in server mode, e.g. in my case, the automation is triggered in local and started by a system service installed in the server.
pywinauto.findwindows.ElementNotFoundError:
{'title': '<my-wanted-title>', 'backend': 'win32', 'visible_only': False}
It is finally solved with another python third-party library pywin32, so providing a backup solution here.
'''
Keep finding message box with specified title and clicking button to close it,
until stopped by the main thread.
'''
import time
from threading import Thread, Event
import win32gui
import win32con
class ButtonClicker(Thread):
def __init__(self, title:str, interval:int):
Thread.__init__(self)
self._title = title
self._interval = interval
self._stop_event = Event()
def stop(self):
'''Stop thread.'''
self._stop_event.set()
#property
def stopped(self):
return self._stop_event.is_set()
def run(self):
while not self.stopped:
try:
time.sleep(self._interval)
self._close_msgbox()
except Exception as e:
print(e, flush=True)
def _close_msgbox(self):
# find the top window by title
hwnd = win32gui.FindWindow(None, self._title)
if not hwnd: return
# find child button
h_btn = win32gui.FindWindowEx(hwnd, None,'Button', None)
if not h_btn: return
# show text
text = win32gui.GetWindowText(h_btn)
print(text)
# click button
win32gui.PostMessage(h_btn, win32con.WM_LBUTTONDOWN, None, None)
time.sleep(0.2)
win32gui.PostMessage(h_btn, win32con.WM_LBUTTONUP, None, None)
time.sleep(0.2)
if __name__=='__main__':
t = ButtonClicker('Configuration Error', 3)
t.start()
time.sleep(10)
t.stop()
i am trying to make a game where players create their own buildings and can then save them for other players to see and play on. However, roblox doesn't let me store all the data needed for the whole creation(there are several properties for each brick)
All i get is this error code:
104: Cannot store Array in DataStore
any help would be greatly appreciated!
I'm not sure if this is the best method, but it's my attempt. Below is an example of a table, you can use tables to store several values. I think you can use HttpService's JSONEncode function to convert tables into strings (which hopefully can be saved more efficiently)
JSONEncode (putting brick's data into a string, which you can save into the DataStore
local HttpService = game:GetService("HttpService")
-- this is an example of what we'll convert into a json string
local exampleBrick = {
["Size"] = Vector3.new(3,3,3),
["Position"] = Vector3.new(0,1.5,0),
["BrickColor"] = BrickColor.new("White")
["Material"] = "Concrete"
}
local brickJSON = HttpService:JSONEncode(exampleBrick)
print(brickJSON)
-- when printed, you'll get something like
-- { "Size": Vector3.new(3,3,3), "Position": Vector3.new(0,1.5,0), "BrickColor": BrickColor.new("White"), "Material": "Concrete"}
-- if you want to refer to this string in a script, surround it with two square brackets ([[) e.g. [[{"Size": Vector3.new(3,3,3)... }]]
JSONDecode (reading the string and converting it back into a brick)
local HttpService = game:GetService("HttpService")
local brickJSON = [[ {"Size": Vector3.new(3,3,3), "Position": Vector3.new(0,1.5,0), "BrickColor": BrickColor.new("White"), "Material": "Concrete"} ]]
function createBrick(tab)
local brick = Instance.new("Part")
brick.Parent = <insert parent here>
brick.Size = tab[1]
brick.Position= tab[2]
brick.BrickColor= tab[3]
brick.Material= tab[4]
end
local brickData = HttpService:JSONDecode(brickJSON)
createBrick(brickData) --this line actually spawns the brick
The function can also be wrapped in a pcall if you want to account for any possible datastore errors.
Encoding a whole model into a string
Say your player's 'building' is a model, you can use the above encode script to convert all parts inside a model into a json string to save.
local HttpService = game:GetService("HttpService")
local StuffWeWantToSave = {}
function getPartData(part)
return( {part.Size,part.Position,part.BrickColor,part.Material} )
end
local model = workspace.Building --change this to what the model is
local modelTable = model:Descendants()
for i,v in pairs(modelTable) do
if v:IsA("Part") or v:IsA("WedgePart") then
table.insert(StuffWeWantToSave, HttpService:JSONEncode(getPartData(modelTable[v])))
end
end
Decoding a string into a whole model
This will probably occur when the server is loading a player's data.
local HttpService = game:GetService("HttpService")
local SavedStuff = game:GetService("DataStoreService"):GetDataStore("blabla") --I don't know how you save your data, so you'll need to adjust this and the rest of the scripts (as long as you've saved the string somewhere in the player's DataStore)
function createBrick(tab)
local brick = Instance.new("Part")
brick.Parent = <insert parent here>
brick.Size = tab[1]
brick.Position= tab[2]
brick.BrickColor= tab[3]
brick.Material= tab[4]
end
local model = Instance.new("Model") --if you already have 'bases' for the players to load their stuff in, remove this instance.new
model.Parent = workspace
for i,v in pairs(SavedStuff) do
if v[1] ~= nil then
CreateBrick(v)
end
end
FilteringEnabled
If your game uses filteringenabled, make sure that only the server handles saving and loading data!! (you probably already knew that) If you want the player to save by clicking a gui button, make the gui button fire a RemoteFunction that sends their base's data to the server to convert it to a string.
BTW I'm not that good at scripting so I've probably made a mistake somehwere.. good luck though
Crabway's answer is correct in that the HttpService's JSONEncode and JSONDecode methods are the way to go about tackling this problem. As it says on the developer reference page for the DataStoreService, Data is ... saved as a string in data stores, regardless of its initial type. (https://developer.roblox.com/articles/Datastore-Errors.) This explains the error you received, as you cannot simply push a table to the data store; instead, you must first encode a table's data into a string using JSONEncode.
While I agree with much of Crabway's answer, I believe the function createBrick would not behave as intended. Consider the following trivial example:
httpService = game:GetService("HttpService")
t = {
hello = 1,
goodbye = 2
}
s = httpService:JSONEncode(t)
print(s)
> {"goodbye":2,"hello":1}
u = httpService:JSONDecode(s)
for k, v in pairs(u) do print(k, v) end
> hello 1
> goodbye 2
As you can see, the table returned by JSONDecode, like the original, uses strings as keys rather than numeric indices. Therefore, createBrick should be written something like this:
function createBrick(t)
local brick = Instance.new("Part")
brick.Size = t.Size
brick.Position = t.Position
brick.BrickColor = t.BrickColor
brick.Material = t.Material
-- FIXME: set any other necessary properties.
-- NOTE: try to set parent last for optimization reasons.
brick.Parent = t.Parent
return brick
end
As for encoding a model, calling GetChildren would produce a table of the model's children, which you could then loop through and encode the properties of everything within. Note that in Crabway's answer, he only accounts for Parts and WedgeParts. You should account for all parts using object:IsA("BasePart") and also check for unions with object:IsA("UnionOperation"). The following is a very basic example in which I do not store the encoded data; rather, I am just trying to show how to check the necessary cases.
function encodeModel(model)
local children = model:GetChildren()
for _, child in ipairs(children) do
if ((child:IsA("BasePart")) or (child:IsA("UnionOperation"))) then
-- FIXME: encode child
else if (child:IsA("Model")) then
-- FIXME: using recursion, loop through the sub-model's children.
end
end
return
end
For userdata, such as Vector3s or BrickColors, you will probably want to convert those to strings when you go to encode them with JSONEncode.
-- Example: part with "Brick red" BrickColor.
color = tostring(part.BrickColor)
print(string.format("%q", color))
> "Bright red"
I suggest what #Crabway said, use HttpService.
local httpService = game:GetService("HttpService")
print(httpService:JSONEncode({a = "b", b = "c"}) -- {"a":"b","b":"c"}
But if you have any UserData values such as Vector3s, CFrames, Color3s, BrickColors and Enum items, then use this library by Defaultio. It's actually pretty nice.
local library = require(workspace:WaitForChild("JSONWithUserdata"))
library:Encode({Vector3.new(0, 0, 0)})
If you want a little documentation, then look at the first comment in the script:
-- Defaultio
--[[
This module adds support for encoding userdata values to JSON strings.
It also supports lists which skip indices, such as {[1] = "a", [2] = "b", [4] = "c"}
Userdata support is implemented by replacing userdata types with a new table, with keys _T and _V:
_T = userdata type enum (index in the supportedUserdataTypes list)
_V = a value or table representing the value
Follow the examples bellow to add suppport for additional userdata types.
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Usage example:
local myTable = {CFrame.new(), BrickColor.Random(), 4, "String", Enum.Material.CorrodedMetal}
local jsonModule = require(PATH_TO_MODULE)
local jsonString = jsonModule:Encode(myTable)
local decodedTable = jsonModule:Decode(jsonString)
--]]
I have an Excel template file . Based on the Excel version, I would like to SaveAs as Temp.xlsm through Matlab.
Here is the code I am using to save the template file :
if(XLversion >= 12.0)
Workbook = invoke(Excel.Workbooks,'Open',tempxls);
tempxls1 = [pwd '\utils\temp.xlsm'];
Workbook.SaveAs(tempxls1,1);
tempxls = tempxls1;
extn = 'xlsm';
end
The code is working fine. However when I try to open the file, I get the following error :
However, when the save the temp.xls to temp.xlsm through SaveAs menu, it opens without any error.
Any idea what could be the error in the code or If I am missing something.
Thanks
Can you try the following code please and comment. You may call this sub-routine to save the as the file.
Private Sub Workbook_BeforeSave(ByVal SaveAsUI As Boolean, Cancel As Boolean)
Dim FileNameVal As String
If SaveAsUI Then
FileNameVal = Application.GetSaveAsFilename(, "Excel Macro-Enabled Workbook (*.xlsm), *.xlsm")
Cancel = True
If FileNameVal = "False" Then 'User pressed cancel
Exit Sub
End If
Application.EnableEvents = False
If Right(ThisWorkbook.Name, 5) <> ".xlsm" Then
ThisWorkbook.SaveAs Filename:=FileNameVal & ".xlsm", FileFormat:=xlOpenXMLWorkbookMacroEnabled
Else
ThisWorkbook.SaveAs Filename:=FileNameVal, FileFormat:=xlOpenXMLWorkbookMacroEnabled
End If
Application.EnableEvents = True
End If
End Sub
Reference:
I want to test Restful web service in SoapUI. For that, I need to read values from Excel and pass that to request.
I searched in net, I found that it is possible through DataGen TestStep. I have SoapUI, but I couldn't find that option.
Can someone please tell if DataGen TestStep is available in SoapUI-4.5.1 or SoapUI Pro.
I am 99% sure that the data sources and such are only in SoapUI pro. You can accomplish the same thing in groovy scripts, though, but you would probably be better off reading from a text file as opposed to a spreadsheet.
The step is available only in Soap UI Pro only(Ready API)
In free version that is Soap UI, you can use POI way of reading an excel file via groovy scripting and pass those values in your input request.
import org.apache.poi.xssf.usermodel.*
import org.apache.poi.ss.usermodel.DataFormatter;
def fs = new FileInputStream("F:\\Gaurav\\soapui\\readFile.xlsx")
def wb = new XSSFWorkbook(fs)
def ws = wb.getSheet("Sheet1")
def r = ws.getPhysicalNumberOfRows()
for(def i =0 ; i < r ; i++)
{
def row = ws.getRow(i);
def c=row.getPhysicalNumberOfCells()
for(def j=0; j <c ; j++)
{
def cell= row.getCell(j)
// to convert everything to a String format
DataFormatter formatter = new DataFormatter()
def cellValue=formatter.formatCellValue(cell)
log.info cellValue
}
}
// Above is the code to read from excel. Once you have read the values, you can // store the values in property
testRunner.testCase.setPropertyValue(tcprop,"cellValue")
then in your request you can expand it like below
${#TestCase#tcprop}
This way you can achieve the same DataGen thing in Soap UI free version 4.5
So, there is an option in SoapUI Setup Script that could be run in advance
You can convert your Excel to csv or text file and handle the date from there.
I've done some testing with REST services, used only reading from the text file feature.
the code like this:
//Load the text file
def inputFile = new File("C://Temp//whatever");
//Create an empty list...
def mega_List = [];
//...and then populate it with the contents
// of the text file.
addSomeThingToList = {mega_List.add(it)};
inputFile.eachLine(addSomeThingToList);
//...and assign its value to the Test Case Property
def tc = testRunner.testCase;
//Randomly pick an item from the list...
def index = context.expand( '${#TestCase#index}' ).toInteger()
if ( index < mega_List.size() ) {
def id = mega_List.get(index);
index++
tc.setPropertyValue("id", id);
tc.setPropertyValue("index", index.toString());
}
else {
tc.setPropertyValue("index", "0");
tc.setPropertyValue("id", "0");
testrunner.cancel( "time to go home" )
}