Comparing results

Bayesian analysis of blinking and bleaching

Comparing results

Postby hoha » Mon Apr 14, 2014 11:00 pm

Dear Susan,

We're comparing the results imagej plugin and the results of the compiled program, we're using the same configurations on the mustilspot5.cfg but the results aren't the same, the later reconstruction seems to have less quality. The version of 3B is 1.1 in the two cases. Could you please advise us about it?

Another question, I'm just wondering if the 'iterations' shown the result file of imagej plugin are the same as the 'iterations' that are shown in the result file of the compiled program, i.e. does the algorithm needs to be stopped by the "Iteration: 50(200)"?

Looking forward...
Haydee O. Hdez. Aviña
Laboratorio Nacional de Microscopía Avanzada
Instituto de Biotecnología
Universidad Nacional Autónoma de México
hoha
 
Posts: 9
Joined: Mon Apr 14, 2014 9:24 pm

Re: Comparing results

Postby edrosten » Fri Apr 18, 2014 3:57 pm

hoha wrote:We're comparing the results imagej plugin and the results of the compiled program, we're using the same configurations on the mustilspot5.cfg but the results aren't the same, the later reconstruction seems to have less quality. The version of 3B is 1.1 in the two cases. Could you please advise us about it?


The code used by the two systems is identical. However there are a few ways for differences to creep in in the configuration.

One is in the spot shape priors, which is relatively tricky to calculate correctly. If you're running the commandline program, this set with the
Code: Select all
 blur.mu
parameter. This value is computed automatically in the plugin using the pixel size and FWHM.

If you open up the output file from the plugin, you can see the complete configuration file embedded very near the beginning like this:

Code: Select all
LOGVERSION 2
PIXELS 26 24 27 24 28 24 29 24 30 24 31 24 32 24 25 25 26 25 27 25 28 25 29 25 30 25 31 25 32 25 33 25 24 26 25 26 26 26 27 26 28 26 29 26 30 26 31 26 32 26 33 26 34 26 24 27 25 27 26 27 27 27 28 27 29 27 30 27 31 27 32 27 33 27 34 27 24 28 25 28 26 28 27 28 28 28 29 28 30 28 31 28 32 28 33 28 34 28 24 29 25 29 26 29 27 29 28 29 29 29 30 29 31 29 32 29 33 29 34 29 24 30 25 30 26 30 27 30 28 30 29 30 30 30 31 30 32 30 33 30 34 30 24 31 25 31 26 31 27 31 28 31 29 31 30 31 31 31 32 31 33 31 34 31 24 32 25 32 26 32 27 32 28 32 29 32 30 32 31 32 32 32 33 32 34 32 25 33 26 33 27 33 28 33 29 33 30 33 31 33 32 33 33 33 26 34 27 34 28 34 29 34 30 34 31 34 32 34
BEGINGVARLIST
//Registered GVars:
A=[ 1.60000000000000003331e-01 8.39999999999999968914e-01 0.00000000000000000000e+00; 4.94999999999999995559e-01
4.94999999999999995559e-01 1.00000000000000002082e-02; 0.00000000000000000000e+00 0.00000000000000000000e+00 1.00
000000000000000000e+00]
add_remove.hessian.inner_samples=1000
add_remove.hessian.outer_samples=100
add_remove.optimizer.attempts=10
add_remove.optimizer.hessian_inner_samples=1000
add_remove.optimizer.samples=20
add_remove.thermo.samples=1000
add_remove.tries=10
blur.mu=2.90044268048142116212e-02
blur.sigma=1.00000000000000005551e-01
cg.max_motion=5.00000000000000000000e-01
gibbs.mixing_iterations=1
...


Check that the blur.mu is the same in both files.

It's also necessary to ensure that the areas used for evaluation are the same. This is encoded in the PIXELS line above. If those are identical between the commandline program and plugin then you're using the same area.

Finally, for the results to be truly identical, the random seeds must be the same. This is another configuration parameter so will be embedded in the file. Generally the different seeds shouldn't affect the final result much.

Let me know if that fixes the problem.

Another question, I'm just wondering if the 'iterations' shown the result file of imagej plugin are the same as the 'iterations' that are shown in the result file of the compiled program, i.e. does the algorithm needs to be stopped by the "Iteration: 50(200)"?


The iterations in the plugin correspond to the larger of the two numbers in the commandline program. That's generally regarded as a safe minimum. It should be OK to leave it running longer.
edrosten
 
Posts: 42
Joined: Wed Jul 18, 2012 2:54 pm

Re: Comparing results

Postby hoha » Thu May 01, 2014 9:23 pm

Thank you!
hoha
 
Posts: 9
Joined: Mon Apr 14, 2014 9:24 pm

Re: Comparing results

Postby rabia » Thu Oct 23, 2014 6:05 am

Dear Susan,

We're comparing the results imagej plugin and the results of the compiled program, we're using the same configurations on the mustilspot5.cfg but the results aren't the same, the later reconstruction seems to have less quality. The version of 3B is 1.1 in the two cases. Could you please advise us about it?

Another question, I'm just wondering if the 'iterations' shown the result file of imagej plugin are the same as the 'iterations' that are shown in the result file of the compiled program, i.e. does the algorithm needs to be stopped by the "Iteration: 50(200)"?
awais
rabia
 
Posts: 1
Joined: Thu Oct 23, 2014 6:00 am

Re: Comparing results

Postby edrosten » Thu Oct 30, 2014 3:44 pm

rabia wrote:We're comparing the results imagej plugin and the results of the compiled program, we're using the same configurations on the mustilspot5.cfg but the results aren't the same, the later reconstruction seems to have less quality. The version of 3B is 1.1 in the two cases. Could you please advise us about it?


The results should be the same, however, it's easy for an error to creep in. All the parameters for the analysis are stored at the beginning of the output file, in alphabetical order. The parameters start at the beginning of the file and continue until the ENDGVARLIST. They will look something like this:

Code: Select all
LOGVERSION 2
PIXELS 26 24 27 24 28 24 29 24 30 24 31 24 32 24 25 25 26 25 27 25 28 25 29 25 30 25 31 25 32 25 33 25 24 26 25 26 26 26 27 26 28 26 29 26 30 26 31 26 32 26 33 26 34 26 24 27 25 27 26 27 27 27 28 27 29 27 30 27 31 27 32 27 33 27 34 27 24 28 25 28 26 28 27 28 28 28 29 28 30 28 31 28 32 28 33 28 34 28 24 29 25 29 26 29 27 29 28 29 29 29 30 29 31 29 32 29 33 29 34 29 24 30 25 30 26 30 27 30 28 30 29 30 30 30 31 30 32 30 33 30 34 30 24 31 25 31 26 31 27 31 28 31 29 31 30 31 31 31 32 31 33 31 34 31 24 32 25 32 26 32 27 32 28 32 29 32 30 32 31 32 32 32 33 32 34 32 25 33 26 33 27 33 28 33 29 33 30 33 31 33 32 33 33 33 26 34 27 34 28 34 29 34 30 34 31 34 32 34
BEGINGVARLIST
//Registered GVars:
A=[ 1.60000000000000003331e-01 8.39999999999999968914e-01 0.00000000000000000000e+00; 4.94999999999999995559e-01 4.94999999999999995559e-01 1.00000000000000002082e-02; 0.00000000000000000000e+00 0.00000000000000000000e+00 1.00000000000000000000e+00]
add_remove.hessian.inner_samples=1000
add_remove.hessian.outer_samples=100
add_remove.optimizer.attempts=10
add_remove.optimizer.hessian_inner_samples=1000
add_remove.optimizer.samples=20
add_remove.thermo.samples=1000
add_remove.tries=10
blur.mu=2.90044268048142116212e-02
blur.sigma=1.00000000000000005551e-01
cg.max_motion=5.00000000000000000000e-01
gibbs.mixing_iterations=1
intensity.rel_mu=2.00000000000000000000e+00
intensity.rel_sigma=1.00000000000000000000e+00
main.cg.max_iterations=5.00000000000000000000e+00
main.gibbs.samples=10
main.passes=4
main.total_iterations=100000000
max_motion.use_brightness_std=1
mode="new"
pi=[ 5.00000000000000000000e-01 5.00000000000000000000e-01 0.00000000000000000000e+00 ]
placement="intensity_sampled"
placement.uniform.num_spots=11
position.extra_radius=2.29999999999999982236e+00
position.use_prior=1
preprocess.lpf=5.00000000000000000000e+00
preprocess.skip=0
seed=0
//Unmatched tags:
add_remove.gibbs.samples=1
add_remove.max_iterations=10
add_remove.max_tries=10
cluster_to_show=19
debug.zoom=4
edge=1000
log_ratios_zoom=[1 1]
main.cg.max_tries=10
preprocess.fixed_scaling=0
radius=0
threshold=0
use_largest=1
// End of GVar list.
ENDGVARLIST


Can you extract the two parameter sets from the two files and compare them to see if they are the same? Some differences may have crept in.

The settings might not be in the same order, which may make things harder (the order is not important to the program). If you have two files (results1.txt and results2.txt), and you run the following command from the commandline in the directory where those two files are:

diff <( sed '/ENDGVAR/q' results1.txt | sort ) <( sed '/ENDGVAR/q' results2.txt | sort )

Then, it should show you how the two configurations differ.

Another question, I'm just wondering if the 'iterations' shown the result file of imagej plugin are the same as the 'iterations' that are shown in the result file of the compiled program, i.e. does the algorithm needs to be stopped by the "Iteration: 50(200)"?


The iterations shown in the plugin correspond to the large of the two numbers (the one in brackets). It should often be safe to stop the run by that point.
edrosten
 
Posts: 42
Joined: Wed Jul 18, 2012 2:54 pm


Return to 3B

Who is online

Users browsing this forum: No registered users and 1 guest

cron