I am trying to apply the N4ITKBiasFieldCorrectionImageFilter from SimpleITK, to an MRI scan as a pre-processing step. From what I've gathered this can be executed in 3D and I've been applying it with this code:
corrector = sitk.N4BiasFieldCorrectionImageFilter()
maskImage=sitk.OtsuThreshold(Original_scan, 0, 1, 200)
I = sitk.Cast( Original_scan, sitk.sitkFloat32 )
BiasCorrected_scan=corrector.Execute(I,maskImage)
I keep getting the same Error eventually as it executes the function:
RuntimeError: Exception thrown in SimpleITK N4BiasFieldCorrectionImageFilter_Execute: c:\d\vs14-win64-pkg\simpleitk-build\itk-prefix\include\itk-4.11\itkImageToImageFilter.hxx:250:
itk::ERROR: SubtractImageFilter(0000021C00CCEEA0): Inputs do not occupy the same physical space!
InputImage Origin: [1.2708700e+02, -1.5783400e+02, 9.0149000e+01], InputImage_1 Origin: [6.5250000e+01, -5.4894173e+01, 5.4894173e+01]
Tolerance: 4.2969999e-05
InputImage Spacing: [4.2969999e-01, 4.2969999e-01, 3.0000000e+00], InputImage_1 Spacing: [5.4894173e+01, 5.4894173e+01, 6.5250000e+01]
Tolerance: 4.2969999e-05
Does anybody have experience executing this filter?
PS. even if I set origin and spacing before executing the filter, the error happens.
Thanks ahead of time
That should not happen. Your data is either odd some how or the is a small bug in the filter.
A quick hack would be to call sitk::ProcessObject::SetGlobalDefaultCoordinateTolerance (double) to a large enough number as to hide this error.
try to set defined MaximunNumberOfIterations:::
corrector = sitk.N4BiasFieldCorrectionImageFilter()
numberFittingLevels = 4
numberOfIteration = [50] (array)
corrector.SetMaximumNumberOfIterations(numberOfIteration * numberFittingLevels)
set lower values till your tolerance Value is accepted
Related
When trying to create a table with the conditional random effects in r using the gtsummary function tbl_regression from a glmmTMB mixed effects negative-binomial zero-inflated model, I get duplicate random effects rows.
Example (using Mollie Brooks' Zero-Inflated GLMMs on Salamanders Dataset):
data(Salamanders)
head(Salamanders)
library(glmmTMB)
zinbm2 = glmmTMB(count~spp + mined +(1|site), zi=~spp + mined + (1|site), Salamanders, family=nbinom2)
zinbm2_table_cond <- tbl_regression(
zinbm2,
tidy_fun = function(...) broom.mixed::tidy(..., component = "cond"),
exponentiate = TRUE,
estimate_fun = purrr::partial(style_ratio, digits = 3),
pvalue_fun = purrr::partial(style_sigfig, digits = 3))
zinbm2_table_cond
Output:
Random Effects Output (cond)
When extracting the random effects from de zero-inflated part of the model I get the same problem.
Example:
zinbm2_table_zi <- tbl_regression(
zinbm2,
tidy_fun = function(...) broom.mixed::tidy(..., component = "zi"),
exponentiate = TRUE,
estimate_fun = purrr::partial(style_ratio, digits = 3),
pvalue_fun = purrr::partial(style_sigfig, digits = 3))
zinbm2_table_zi
Output:
Random Effects Output (zi)
The problem persists if I specify the effects argument in broom.mixed.
tidy_fun = function(...) broom.mixed::tidy(..., effects = "ran_pars", component = "cond"),
Looking at confidence intervals in both outputs it seems that somehow it is extracting random effects from both parts of the model and changing the estimate of the zero-inflated random effects (in 1st image; opposite in the 2nd image) to match the conditional part estimate while keeping the CI.
I am not knowledgeable enough to understand why this is happening. Since both rows have the same label I am having difficulty removing the wrong one.
Any tips on how to avoid this problem or a workaround to remove the undesired rows?
If you need more info, let me know.
Thank you in advance.
PS: Output images were changed to link due to insufficient reputation.
I am working with Dymola, and try to use the functions provided by Modelica standard library in the command window, but it seems that I can't use them, and I couldn't claim a variable of a specific type either. I am wondering if there is some kind of limit of the command I could use in the command window of Dymola. Where should I find all the allowable commands?
I try to use some functions from Modelica.Media, it seems the input variables are out of range, but I tried a lot of times and different units system. I find that I can't declare a variable of pressure type in the command window, but Modelica.Media.Water.IF97_Utilities.h_pT() requires that I need to provide the variable as pressure and enthalpy type, is this the reason I can't use this function in the command window?
Modelica.Media.Water.IF97_Utilities.h_pT(1e6,800,1)
Failed to expand Modelica.Media.Water.IF97_Utilities.h_props_pT(
1000000.0,
800,
Modelica.Media.Common.IF97BaseTwoPhase(
phase = 1,
region = 1,
p = 1000000.0,
T = 800.0,
h = 9.577648835649013E+20,
R = 461.526,
cp = 1.8074392528071426E+20,
cv = -3.7247229288028774E+18,
rho = 5.195917767496603E-13,
s = 1.2052984524009106E+18,
pt = 645518.9415389205,
pd = 6.693617079374418E+18,
vt = 357209983199.2206,
vp = -553368.7088215105,
x = 0.0,
dpT = 645518.9415389205
)).
Failed to expand Modelica.Media.Water.IF97_Utilities.h_pT(1000000.0, 800, 1).
Assuming the inputs are valid there seems to be an issue specifically related to evaluating some media-functions interactively in Dymola (since they shouldn't be evaluated in models). It will be corrected in Dymola 2022x.
A temporary work-around is to first set the flag Advanced.SemiLinear = false; and then:
Modelica.Media.Water.IF97_Utilities.h_pT(1e6,800,1)
= 9.577648835649013E+20
(I'm not sure how valid the formulation is in that region.)
But please remember to set Advanced.SemiLinear = true; before translating and simulating any models - in particular models using media-functions.
The problem is that you are giving the function an invalid input. It seems Dymola does not give you the error-message for this based on the screenshot and logs you provided. I tried it in OpenModelica and got:
Modelica.Media.Water.IF97_Utilities.h_pT(100e5, 500e3)
[Modelica 4.0.0/Media/Water/IF97_Utilities.mo:2245:9-2246:77] Error: assert triggered: IF97 medium function g5: input temperature (= 500000 K) is higher than limit of 2273.15K in region 5
By using a value within the limits, it returns a value:
Modelica.Media.Water.IF97_Utilities.h_pT(100e5, 1e3)
Result
Result2
Solved
Dealing with low amounts of data, and dealing with overfitting w/ Folding[GridSearchCV]
I am completely stumped as to how to get better estimations from my model. It seems that when I try to run my code, I get negative Accuracies. How can I improve cross_val_score or testing scores or whatever you want to call it such that I can predict values more reliably.
I tried adding more data (from 50 to 200+).
I tried random parameters (and realized this was a Naive approach)
I also tried cleaning my data w/ StandardScaler on the features
Anyone have any suggestions?
from sklearn.neural_network import MLPRegressor
from sklearn import preprocessing
import requests
import json
from calendar import monthrange
import numpy as np
from sklearn.model_selection import cross_val_score, GridSearchCV
from sklearn.preprocessing import scale
r =requests.get('https://www.alphavantage.co/query?function=TIME_SERIES_WEEKLY_ADJUSTED&symbol=W&apikey=QYQ2D6URDOKNUGF4')
#print(r.text)
y = json.loads(r.text)
#print(y["Monthly Adjusted Time Series"].keys())
keysInResultSet = y["Weekly Adjusted Time Series"].keys()
#print(keysInResultSet)
featuresListTemp = []
labelsListTemp = []
count = 0;
for i in keysInResultSet:
#print(i)
count = count + 1;
#print(y["Monthly Adjusted Time Series"][i])
tmpList = []
tmpList.append(count)
featuresListTemp.append(tmpList)
strValue = y["Weekly Adjusted Time Series"][i]["5. adjusted close"]
numValue = float(strValue)
labelsListTemp.append(numValue)
print("TOTAL SET")
print(featuresListTemp)
print(labelsListTemp)
print("---")
arrTestInput = []
arrTestOutput = []
print("SCALING SET")
X_train = np.array(featuresListTemp)
scaler = preprocessing.StandardScaler().fit(X_train)
X_train_scaled = scaler.transform(X_train)
print(X_train_scaled)
product_model = MLPRegressor()
#10.0 ** -np.arange(1, 10)
#todo : once found general settings, iterate through some more seeds to find one that can be used on the training
parameters = {'learning_rate': ['constant','adaptive'],'solver': ['lbfgs','adam'], 'tol' : 10.0 ** -np.arange(1, 4), 'verbose' : [True], 'early_stopping': [True], 'activation' : ['tanh','logistic'], 'learning_rate_init': 10.0 ** -np.arange(1, 4), 'max_iter': [4000], 'alpha': 10.0 ** -np.arange(1, 4), 'hidden_layer_sizes':np.arange(1,11), 'random_state':np.arange(1, 3)}
clf = GridSearchCV(product_model, parameters, n_jobs=-1)
clf.fit(X_train_scaled, labelsListTemp)
print(clf.score(X_train_scaled, labelsListTemp))
print(clf.best_params_)
best_params = clf.best_params_
newPM = MLPRegressor(hidden_layer_sizes=((best_params['hidden_layer_sizes'])), #try reducing the layer size / increasing it and playing around with resultFit variable
batch_size='auto',
power_t=0.5,
activation=best_params['activation'],
solver=best_params['solver'], #non scaled input
learning_rate=best_params['learning_rate'],
max_iter=best_params['max_iter'],
learning_rate_init=best_params['learning_rate_init'],
alpha=best_params['alpha'],
random_state=best_params['random_state'],
early_stopping=best_params['early_stopping'],
tol=best_params['tol'])
scores = cross_val_score(newPM, X_train_scaled, labelsListTemp, cv=10, scoring='neg_mean_absolute_error')
print("Accuracy: %0.2f (+/- %0.2f)" % (scores.mean(), scores.std() * 2))
print(scores)
Output from line 63 and down
0.9142644531564619 {'activation': 'logistic', 'alpha': 0.001, 'early_stopping': True, 'hidden_layer_sizes': 7, 'learning_rate':
'constant', 'learning_rate_init': 0.1, 'max_iter': 4000,
'random_state': 2, 'solver': 'lbfgs', 'tol': 0.01, 'verbose': True}
Accuracy: -21.91 (+/- 58.89) [ -32.87854574 -105.0632913
-22.89836453 -7.33154414 -22.38773819 -3.3786339 -1.7658796 -3.78002866 -4.78734308 -14.81212738]
{'activation': 'logistic', 'alpha': 0.01, 'early_stopping': True, 'hidden_layer_sizes': 30, 'learning_rate': 'constant', 'learning_rate_init': 0.1, 'max_iter': 4000, 'random_state': 2, 'solver': 'lbfgs', 'tol': 0.1, 'verbose': True}
{'activation': 'tanh', 'alpha': 0.01, 'early_stopping': True, 'hidden_layer_sizes': 99, 'learning_rate': 'constant', 'learning_rate_init': 0.1, 'max_iter': 4000, 'random_state': 1, 'solver': 'lbfgs', 'tol': 0.01, 'verbose': True}
Both configurations stated above will work for the sample set. Thanks all, please let me know if there are any questions. This can be solved by scaling down all your other parameters ie. instead of 10.0 ** -np.arange(1, 3) do 10.0 ** -np.arange(1, 2)
to a more limited set. Start removing parameters that you know are correct (very hard to do, but one could be learning_rate='constant' as I noticed that all my best fits resulted in a learning rate that was constant, regardless of any other parameters.
This is mostly for time optimization but will also help with overfitting as you increase the number of nodes in the network. The idea is that you want to increase the fit some N degrees without losing too much of the generalization properties of the true function) once you perform your first grid search.
You should start you grid search making sure that the # of hidden nodes is some where between the # of input nodes and the # of output nodes.
Once you find a decent fit, you can improve the fit by increasing the number of nodes. You must take care not to add too many nodes as to lose the generalization power of the true function. Before you even start thinking about scaling up, you must start reducing the complexity of the parameters such that on your second grid search you will be performing it on an increased number of nodes w/ more general parameters.
The generalization of parameters is described above with the second grid search taking into account more general parameters from the initial search, whilst increasing the network nodes.
I know this is confusing but it's what helped me fit this decently.
For anyone struggling I would try to
0) generalize after performing a search and getting a decent model
1) use generalization on second search with increased nodes
2) play with alpha parameter while scaling up (the rest of the parameters you can generalize)
3) add a few different seeds or remove them depending on the situation
4) While changing tol will alter fit it is also highly dependent on the number of iterations. For that reason, depending on the case, a reasonable number might be .01 or .001 (reasonable depending on how many iterations you want to wait for a given result/ opportunity to converge) If the tol is set too low, you will run out of iterations as each epoch will never get a chance to stop early.
I am trying to access SCNGeometrySource content. After inspecting the source I can tell that:
source.bytesPerComponent = 4
source.componentsPerVector = 3
source.dataStride = 16
source.dataOffset = 0
source.vectorCount = 98304
In other terms, I have 98304 vectors of float3.
Yet, when calling
source.data.withUnsafeBytes { (pointer: UnsafePointer<float3>) in
for i in 0..<source.vectorCount {
print((pointer + i).pointee)
}
}
The first printed elements seem about fine, but close to the end, I get weird values close to zero when they shouldn't be, crash from time to time, and memory sanitiser does detect a heap overflow.
I don't understand how can a heap overflow being caused even though it looks like I am accessing everything the right way.
What should I do to isolate the cause of this issue?
EDIT (partial solution) :
In my case, I had a custom geometry and used the SCNGeometrySource(vertices: [SCNVector3]) initialiser. As far as I can tell there seems be a bug causing incorrect allocations on what source.data returns. The way around this is not to use the SCNGeometrySource(vertices: [SCNVector3]) initialiser anymore and to instead use the SCNGeometrySource(data: ...) initialiser. This fixed the issue for me.
what do you call Float3? Is it the SIMD type float3? In this case the stride would be 16, not 12 (SIMD types are aligned on 4 bytes).
print(MemoryLayout<float3>.stride) // 16
We're running into a problem with one of our reports. In one of our tablixes a textbox has the following expression:
=Iif(Fields!SomeField.Value = 0, 0, Fields!SomeOtherField.Value / Fields!SomeField.Value)
Which should be pretty self-explanatory. If "SomeField" is zero, set the text box value to zero, else set it to "SomeOtherValue / SomeValue".
What has us stumped is that the report still throws a runtime exception "attempted to divide by zero" even though the above expression should prevent that from happening.
We fiddled a bit with the expression just to make sure that the zero-check is working, and
=Iif(Fields!SomeField.Value = 0, "Yes", "No")
works beautifully. Cases where the data is in fact zero resulted in the textbox displaying "Yes" and vice versa. So the check works fine.
My gut feel is that the Report rendering engine throws the exception at run-time, because it "looks" as if we are going to divide by zero, but in actual fact, we're not.
Has anyone run into the same issue before? If so, what did you do to get it working?
IIf will always evaluate both results before deciding which one to actually return.
Try
=IIf(Fields!SomeField.Value = 0, 0, Fields!SomeOtherField.Value / IIf(Fields!SomeField.Value = 0, 1, Fields!SomeField.Value))
This will use 1 as the divisor if SomeOtherField.Value = 0, which does not generate an error. The parent IIf will return the correct 0 for the overall expression.
An easy clean way to prevent a divide by zero error is using the report code area.
In the Menu, go to Report > Report Properties > Code and paste the code below
Public Function Quotient(ByVal numerator As Decimal, denominator As Decimal) As Decimal
If denominator = 0 Then
Return 0
Else
Return numerator / denominator
End If
End Function
To call the function go to the the Textbox expression and type:
=Code.Quotient(SUM(fields!FieldName.Value),SUM(Fields!FieldName2.Value))
In this case I am putting the formula at the Group level so I am using sum. Otherwise it would be:
=Code.Quotient(fields!FieldName.Value,Fields!FieldName2.Value)
From: http://williameduardo.com/development/ssrs/ssrs-divide-by-zero-error/
On reflection, I feel best idea is to multiply by value to power -1, which is a divide:
=IIf
(
Fields!SomeField.Value = 0
, 0
, Fields!SomeOtherField.Value * Fields!SomeField.Value ^ -1
)
This doesn't fire pre-render checks as val * 0 ^ -1 results in Infinity, not error
IIF evaluates both expression even thought the value of Fields!SomeField.Value is 0. Use IF instead of IIF will fix the problem.