User Forum of Software BASEMENT

BASEMENT
Basic Simulation Environment for computation of environmental flow and natural hazard simulation
Laboratory of Hydraulics, Hydrology and Glaciology (VAW)
ETH Zurich		Basement_Logo

You are not logged in.

Pages: 1

#1 2020-04-26 19:54:23

Franzaffa
User
Registered: 2017-02-26
Posts: 14

Simulation end after 20%

Hi, here i report an error from Basement V3.02:

HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5F.c line 444 in H5Fcreate(): unable to create file
    major: File accessibilty
    minor: Unable to open file
  #001: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5Fint.c line 1533 in H5F_open(): file exists
    major: File accessibilty
    minor: Unable to open file
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5G.c line 299 in H5Gcreate2(): not a location
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5Gloc.c line 246 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5G.c line 683 in H5Gclose(): not a group
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5G.c line 299 in H5Gcreate2(): not a location
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5Gloc.c line 246 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5G.c line 683 in H5Gclose(): not a group
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5G.c line 299 in H5Gcreate2(): not a location
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5Gloc.c line 246 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5G.c line 683 in H5Gclose(): not a group
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5D.c line 119 in H5Dcreate2(): not a location ID
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5Gloc.c line 246 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5Dio.c line 314 in H5Dwrite(): dset_id is not a dataset ID
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5D.c line 331 in H5Dclose(): not a dataset
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5D.c line 282 in H5Dopen2(): not a location
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5Gloc.c line 246 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5A.c line 263 in H5Acreate2(): not a location
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5Gloc.c line 246 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5A.c line 615 in H5Awrite(): not an attribute
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5A.c line 1523 in H5Aclose(): not an attribute
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5A.c line 263 in H5Acreate2(): not a location
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5Gloc.c line 246 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5A.c line 615 in H5Awrite(): not an attribute
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5A.c line 1523 in H5Aclose(): not an attribute
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5A.c line 263 in H5Acreate2(): not a location
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5Gloc.c line 246 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5A.c line 615 in H5Awrite(): not an attribute
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5A.c line 1523 in H5Aclose(): not an attribute
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5D.c line 331 in H5Dclose(): not a dataset
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
  #000: C:\Users\manuel\Documents\BMv3\externalLibs\vcpkg\buildtrees\hdf5\src\hdf5-368e55e917\hdf5-1.10.5\src\H5F.c line 671 in H5Fclose(): not a file ID
    major: File accessibilty
    minor: Inappropriate type
 -> cannot attach attribute to dataset </RESULTS/CellsAll/HydState/0000034> in file <C:/Users/franz/Desktop/Model for platform/results.h5>, in file: Z:\src\dataio\H5Base.cpp on line 76, version v3.0.2
 -> ... unfortunately BASEMENT is killed!

The simulation end after 20%; if i restart from the last timestep, the simulation works until 20% and after stop another time and so on.
This is the Basement output console:

How can i solve it?
Thanks
Francesco

Offline

#2 2020-05-20 09:40:18

Matthias Bürgler
Developer
From: Zürich
Registered: 2019-04-04
Posts: 175

Re: Simulation end after 20%

Hi Francesco

According to the console output, the simulation crashed because it could not write a simulation result to the results.h5 file. There can be different reasons for this, e.g. running out of space on your hard-drive, changing the result.h5 file name or path (folder name) during the simulation, opening the results.h5 file during the simulation, etc.

I this case, the results.h5 file can become corrupted, especially the dataset which is was trying to write when it crashed "/RESULTS/CellsAll/HydState/0000034". If the -h5 file is corrupted, the simulation will crash again when it tries to write to the corrupted dataset.

To recover data from a corrupted .h5 file is probably possible, e.g. with a python script. But first I would create a copy of the simulation, delete the results.h5 file and restart the simulation to see if it runs past the 20%, while making sure you have enough disk space.

Best regards
Matthias

Offline

#3 2020-05-21 06:38:39

Franzaffa
User
Registered: 2017-02-26
Posts: 14

Re: Simulation end after 20%

Hi, thanks for the answer, i tried to change path etc but nothing change; when the .h5 become more than 2GB the program is unable to write results. The file is not corrupted, only the last timestep is not visible. I can try with, for example, linux systems to verify if it is a OS problem (limit due to the file dimesion). My Hard disk is half empty (more than 200 GB available)
Thanks
Francesco

Offline

#4 2020-05-25 09:52:14

Matthias Bürgler
Developer
From: Zürich
Registered: 2019-04-04
Posts: 175

Re: Simulation end after 20%

Hi Francesco

Franzaffa wrote:

when the .h5 become more than 2GB the program is unable to write results.

Different hard drive formats have different limitations on the file size. For more information visit: https://en.wikipedia.org/wiki/Compariso … le_systems.

Best regards
Matthias

Offline

#5 2020-07-06 16:18:37

Matthias
Advanced_User
Registered: 2014-12-16
Posts: 29

Re: Simulation end after 20%

Hi Matthias, Francesco

I am dealing with the same issue. The results.h5-file is exactly 2GB large as the simulation crashes with the same error as above;
I have an NTFS-formatted hard-drive which permits very very large single files.

Could you further investigate and reproduce this on your system?

Thanks a lot and best regards
PS: If I reduce the # of output timesteps so that the results.h5 stays below 2GB, the simulation runs through.

edit: using BMv3.0.2 on Windows in batch-mode (no GUI).

Last edited by Matthias (2020-07-06 16:19:45)

Offline

#6 2020-07-07 08:39:15

Matteo Facchini
Developer
From: Trento
Registered: 2014-09-05
Posts: 286

Re: Simulation end after 20%

I am facing the same issue: at 2 gb it crashes with the same errors

using BMv3.0.2 on Windows in batch-mode (no GUI)

Offline

#7 2020-07-20 08:38:57

Matthias
Advanced_User
Registered: 2014-12-16
Posts: 29

Re: Simulation end after 20%

Is there a workaround for this issue for a Windows system?

Offline

#8 2020-07-20 11:48:48

Matteo Facchini
Developer
From: Trento
Registered: 2014-09-05
Posts: 286

Re: Simulation end after 20%

I think there is not...unless you want to install an ubuntu guest with a virtual machine on your windows system...but then it is a pain to bridge the graphic card. You might have better luck with a disk partition.

Offline

#9 2020-08-24 08:57:55

Matteo Facchini
Developer
From: Trento
Registered: 2014-09-05
Posts: 286

Re: Simulation end after 20%

Hi, today I'll try to solve this issue.

I found an answer by Sam on an old thread. I'll look into it and update this answer if I find a solution!

Cheers,
Matteo

UPDATE:
nothing new, the 2GB limit appears to be on basement-side since my windows 10 architecture is 64-bit, basement is a 64-bit app and the disk where I am storing the data is NTFS.

Last edited by Matteo Facchini (2020-08-24 14:45:39)

Offline

#10 2020-12-21 13:13:09

Matteo Facchini
Developer
From: Trento
Registered: 2014-09-05
Posts: 286

Re: Simulation end after 20%

Matthias Bürgler wrote:

Hi Francesco

Franzaffa wrote:

when the .h5 become more than 2GB the program is unable to write results.

Different hard drive formats have different limitations on the file size. For more information visit: https://en.wikipedia.org/wiki/Compariso … le_systems.

Best regards
Matthias

Hi Matthias,

do you think you could try and compile a version of basement with the large adress aware flag?

EDIT: for the time being I am doing the following:

  1. clean the results.h5 file with HDF-View (one should go to RESULTS/CellsAll and delete the exceeding results, i.e. if your simulation stops at time step N there should be some of the variables that have saved also the N+1 result: delete these ones)

  2. re-run the simulation from where it stopped


EDIT 2: the workaround does not work. At the moment basement 3.1 is not able to write files larger than 2 GB on windows 10.

Last edited by Matteo Facchini (2020-12-21 13:41:36)

Offline

#11 2021-01-11 08:08:15

qtheiler
User
Registered: 2015-07-30
Posts: 60

Re: Simulation end after 20%

Hi everybody,

Do you know if the developer of Basement will resolve this problem? This is really annoying at this moment.

Cheers,

Quentin

Offline

#12 2021-01-19 15:49:49

Matthias Bürgler
Developer
From: Zürich
Registered: 2019-04-04
Posts: 175

Re: Simulation end after 20%

We will have a look at this. It seems that the problem only occurs on Windows.

Offline

#13 2021-03-05 15:25:59

Matteo Facchini
Developer
From: Trento
Registered: 2014-09-05
Posts: 286

Re: Simulation end after 20%

This issue should have been fixed in the 3.1.1 release.

Thanks base-team!

I'll test it over the week end and report back here on monday!

EDIT: ti works perfectly! Kudos

Last edited by Matteo Facchini (2021-03-08 09:43:28)

Offline

Pages: 1

Board footer

Powered by FluxBB