the OutputStream/InputStream implementations, which was just finished
last weekend with a few tests on Windows and Ubuntu. Hope it could
work and get more feedbacks soon.
I've added a new option "SchemaData" to the osg2 plugin. Developers
may test the new feature with the command line:
# osgconv cow.osg cow.osgb -O SchemaData
It will record all serializer properties used in the scene graph, at
the beginning of the generated file. And when osgviewer and user
applications is going to read the osgb file, the inbuilt data will be
automatically read and applied first, to keep backwards compatibility
partly. This will not affect osgb files generated with older versions.
"
modifications of the osgShadow header naming styles as well. The
osgDB::Serializer header is also changed to add new Vec2 serializer
macros because of the needs of osgShadow classes. It should compile
fine on both Windows and Linux. But I have only done a few tests to
generate .osgb, .osgt and .osgx formats with these new wrappers."
few headers and the osgAnimation sources are also modified to make
everything goes well, including:
A new REGISTER_OBJECT_WRAPPER2 macro to wrap classes like
Skeleton::UpdateSkeleton.
A bug fix in the Seralizer header which avoids setting default values
to objects.
Naming style fixes in osgAnimation headers and sources, also in the
deprecated dotosg wrappers.
A bug fix for the XML support, to write char values correctly.
A small change in the osg::Geometry wrapper to ignore the
InternalGeometry property, which is used by the MorphGeometry and
should not be set by user applications.
The avatar.osg, nathan.osg and robot.osg data files all work fine with
serializers, with some 'unsupported wrapper' warnings when converting.
I'm thinking of removing these warnings by disabling related property
serializers (ComputeBoundingBoxCallback and Drawable::UpdateCallback),
which are seldom recorded by users.
By the way, I still wonder how would we handle the C4121 problem,
discussed some days before. The /Zp compile option is set to 16 in the
attached cmake script file. And is there a better solution now?"
now O(n) rather than O(nlogn) where n is the number of requests. The refactoring also cleans up the access of the
request lists so that the code is more readable/maintainable.
using osgDB::XmlParser. The extension for XML-formatted scenes is
.osgx, corresponding to .osgb for binary and .osgt for ascii. It could
either be rendered in osgviewer or edited by common web browsers and
xml editors because of a range of changes to fit the XML syntax. For
example, the recorded class names are slight modified, from
'osg::Geode' to 'osg--Geode'.
To quickly get an XML file:
# ./osgconv cow.osg cow.osgx
The StreamOperator header, InputStreram and OutputStream classes are
modified to be more portable for triple ascii/binary/XML formats. I
also fixed a bug in readImage()/writeImage() to share image objects if
needed.
The ReaderWriterOSG2 class now supports all three formats and
reading/writing scene objects (not nodes or images), thanks to
Torben's advice before.
"
This change should make it possible to delete PagedLOD's independantly from the DatabasePager, and also prevent issues of
consistency of the pager when subgraphs when are cached elsewhere in the application such as in the Registry filecache.
of OSG. I modified the osgDB::InputStream and OutputStream and the
PagedLOD wrapper as well. Now all seems to work fine with paged
scenes. I've tested with the puget terrain data and the osgdem
application from VPB:
# osgdem --xx 10 --yy 10 -t ps_texture_4k.tif --xx 10 --yy 10 -d
ps_height_4k.tif -l 8 -v 0.1 -o puget.osgb
As the ive plugin does, The PagedLOD wrapper now automatically add the
latest file path to PagedLODs' databasePath member, to help them find
correct child positions. I also changed the image storage strategy of
the OutputStream class, to store them inline by default. The osgt
extension should also work, in case the image files are also written
to the disk.
"
OSGDB_EXPORT macro to RegisterCompressorProxy, and modified the
findCompressor() method to look for custom compressors in libraries
such like osgdb_compressor_name.so, which was described in the wiki
page chapter 2.4."
1. Rewrite the reading/writing exception handlers to work like the ive
plugin exceptions.
2. Write a header writing/checking function in ReaderWriterOSG2.cpp,
which may help decide if the stream is ascii or binary. The
readInputIterator() function will return null pointer if the input
file is nither osgb nor osgt format, which indicates that the old .osg
format could be used here, in case we've merged the two plugins
together.
3. Add a new ForceReadingImage option in the InputStream, which will
allocate an empty image object with the filename if specifed external
image file is missed. It may be useful for format converting in some
cases.
4. Add new osgParticle wrappers, as well as some modification to the
osgParticle headers, for instance, change isEnabled() to getEnabled().
5. Some fixes to the osg serialization wrappers."
destruction of RequestQueue to remove any pointers held in DatabaseRequest attached to the scene graph, and to
prevent their subsequent use in cases where the scene graph is attached to a new DatabasePager.
to the osgDB::Registry. Added a osgDB::Registry::getObjectWrapperManager() for access of this object wrapper manager. This
change centralises the singleton management in osgDB.
Merged the osgDB::GlobalLookUpTable functionality into ObjectWrapperManger to keep down the number of singletons in use.
From Robert Osfield, refactor of Wang Rui's original osg2 into 3 parts - parts placed into osgDB, the ReaderWriter placed into src/osg/Plugin/osg and wrappers into src/osgWrappers/serializers/osg
1. The node type will be set to ATOM on read of <tag prop="..." ... /> type tags.
2. GROUP and NODE are now written using the same code (and not just duplicated code). Also NODE will not be written as an ATOM if it has no children or contents, so you need to set the type to ATOM if you want the <tag ... /> style.
3. You had put the write of "/>" for ATOM after the "return true", so it had no effect... Moved to before the return.
4. ATOM did not write its properties correctly, fixed.
5. As an added bonus, I made the write() method indent the output so it's more readable. It brings a small public interface change but the indent argument has a default value so client code doesn't need to change (if there even is any).
6. Another added bonus, I've simplified the write() method a bit by factoring out the write for children and properties into protected methods."
filenames starting with a dash "-" character from the (std::vector<std::string>&) version
of osgDB::readNodeFiles. Handling of argument strings is properly implemented in the
osgDB::readNodeFiles(osg::ArgumentParser& arguments,const Options* options)
variant, which most code uses. The (std::vector<std::string>&) version is only called by
the osgconv utility, which does its own argument handling and stripping prior to calling
readNodeFiles().
Also, documented this behaviour in the header comments.
I believe this code removal is a meritful change because leavign the code in causes an
unexpected and undocumented behaviour (ignoring any filename starting with a dash) that
could bite users in the future. This behaviour is not needed for existing functionality
because existing code uses other APIs to handle dash-prefixed arguments anyway.
"
only create 376, then the program would hang.
376 * 8MB stack per thread = 3008 MB
The stack size allocated per thread blew the process address stack.
To get more threads you have to specify a smaller per thread stack,
but while the Thread::start says it will limit the stack size to the
smallest allowable stack size, it won't let it be smaller than the
default. I included the limits.h header to use PTHREAD_STACK_MIN as
the minimum stack size.
As for the deadlock, if the pthread_create failed, the new thread
doesn't exist and doesn't call threadStartedBlock.release(), so the
existing thread deadlocks on threadStartedBlock.block(). Only block
if the thread was started."
First Submission email from Gustav:
"This submission adds a --cache option to osgconv and osgviewer that enables setObjectCacheHint(osgDB::Options::CACHE_ALL); It greatly reduces memory usage when a .osg file has lots of external references with ProxyNode:s that points to the same file.
Options are also added to the osg plugin. The code was already mostly implemented but there was no way to change the options.
includeExternalReferences
writeExternalReferenceFiles
A counter is added to keep track if an external file has already been written down to avoid writing the same file over and over again. If it has already been written once then it is not written again.
The counter is added to the Output class in osgDB.
"
Second Submission email from Gustav:
"This is a continuation to my previous submission.
I noticed that the same problem that I fixed in ProxyNode.cpp for the osg plugin (external files being written over and over again) also existed in the ive plugin. I attached a submission where the ive plugin remembers which external files that have already been written and do not write them again."
Changes to the above done by Robert Osfield,
changed command line parameter to --enable-object-cache
changed set/get methods in osgDB::Output and ive/DataOutputStream.cpp to be s/getExternalFileWritten(const std::string&)
cleaned up set up of osgDB::Options.
Introduced a new callback osgDB::FindFileCallback that overrides the default behavior of findDataFile/findLibraryFile.
Introduced support for assigning ReaderWriter::Options directory to PagedLOD.
Introduced new osgDB::FileLocationCallback for assistancing the DatabasePager to know when a file is hosted on a local or remote file system.
Introduced a new FindFileCallback to Registry to compliement the existing ReadFileCallback and WriteFileCallback.
Added support for assign Find, Read and WriteFileCallbacks to osdDB::Options to enable plugins/applications to override the callbacks just for that
read/write call and any nested file operations
Converted Present3D across from using libxml2 to using the new osgDB::XmlNode/XmlNode::Input classes from Xml Parsing.
This changes removes the dependency on libxml2, and allows the present3D application and p3d to work on all platforms.
(http://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg23098.html)
Background: when you access a file over HTTP, you cannot rely on a file extension being present; instead the file's mime-type is conveyed in the HTTP Content-Type response header. This facility adds a mime-type-to-extension map to the registry to handle this.
There are two new osgDB::Registry functions which are pretty self-explanatory:
void addMimeTypeExtensionMapping( mime-type, extension )
ReaderWriter* getReaderWriterForMimeType( mime-type )
I also added the file osgDB/MimeTypes.cpp which houses a hard-coded list of built-in types. I took the list from here (http://www.webmaster-toolkit.com/mime-types.shtml) and then pared it down to include mostly image and video types, editing them to map to existing plugins where possible.
In addition, I updated the CURL plugin to a) install a set of built-in mime-type mappings, and b) use them to look up an extension in the event that the target filename does not have an extension.
Here is a test case. This URL pulls down a JPEG (without a file extension):
osgviewer --image "http://us.maps3.yimg.com/aerial.maps.yimg.com/ximg?v=1.8&s=256&t=a&r=1&x=0&y=0&z=2"
"
Single features are implemented as bits asother enums in ReaderWriter class already does, so that combinations are possible and fast comparison operations are possible
By default all features are enabled.
I have added this virtual method to the dot plugin to get an idea how to use these features.
With this patch osgconv --formats shows an additional line 'features' for each plugin"
"Attached you'll find a proposal for using different
protocols. The idea behind the new code is:
1.) plugins/apps register protocols which they can handle. This is done
via osgDB::Registry::registerProtocol(aProtocolName). Plugins register
supported protocols as usual via ReaderWriter::supportsProtocol(..), the
Registry is updated accordingly.
2.) osgDB::containsServerAddress checks first for an appearance of "://"
in the filename and then checks the protocol against the set of
registered protocols via Registry::isProtocolRegistered(aProtocollName)
3.) the other getServer*-functions changed as well, there's even a
getServerProtocol-function
With these changes filenames/Urls get routed to loaded plugins even with
different protocols than 'http'."
a capping the number of PagedLOD to a sepcified maximum, with pruning of inactive
PagedLOD when the total number of inactive and active PagedLOD goes above the maximum.
To enable the mode set the env var OSG_MAX_PAGEDLOD to a value something like 1000.
New attribute DatabasePager::_expiryFrames sets number of frames a PagedLOD child is kept in memory. The attribute is set with DatabasePager::setExpiryFrames method or OSG_EXPIRY_FRAMES environmental variable.
New attribute PagedLOD::PerRangeData::_
frameNumber contains frame number of last cull traversal.
Children of PagedLOD are expired when time _AND_ number of frames since last cull traversal exceed OSG_EXPIRY_DELAY _AND_ OSG_EXPIRY_FRAMES respectively. By default OSG_EXPIRY_FRAMES = 1 which means that nodes from last cull/rendering
traversal will not be expired even if last cull time exceeds OSG_EXPIRY_DELAY. Setting OSG_EXPIRY_FRAMES = 0 revokes previous behaviour of PagedLOD.
Setting OSG_EXPIRY_FRAMES > 0 fixes problems of children reloading in lazy rendering applications. Required behaviour is achieved by manipulating OSG_EXPIRY_DELAY and OSG_EXPIRY_FRAMES together.
Two interface changes are made:
DatabasePager::updateSceneGraph(double currentFrameTime) is replaced by DatabasePager::updateSceneGraph(const osg::FrameStamp &frameStamp). The previous method is in #if 0 clause in the header file. Robert, decide if You want to include it.
PagedLOD::removeExpiredChildren(double expiryTime, NodeList &removedChildren) is deprecated (warning is printed), when subclassing use PagedLOD::removeExpiredChildren(double expiryTime, int expiryFrame, NodeList &removedChildren) instead. "
multi-threaded paging, where the Pager manages threads of reading local
and http files via seperate threads. This makes it possible to smoothly
browse large databases where parts of the data are locally cached while
others are on a remote server. Previously with this type of dataset
the pager would stall all paging while http requests were being served,
even when parts of the models are still loadable virtue of being in the
local cache.
Also as part of the refactoring the DatabaseRequest are now stored in the
ProxyNode/PagedLOD nodes to facilitate quite updating in the cull traversal,
with the new code avoiding mutex locks and searches. Previous on big
databases the overhead involved in make database requests could accumulate
to a point where it'd cause the cull traversal to break frame. The overhead
now is negligable.
Finally OSG_FILE_CACHE support has been moved from the curl plugin into
the DatabasePager. Eventually this functionality will be moved out into
osgDB for more general usage.
ReaderWriter::ReadResult now has a FILE_REQUEST enum.
ReaderWriter::Options now has a s/getAsynchronousFileReadHint() parameter methods.
libcurl based plugin now detects enabing of the AsynchronousFileReadHint, but
as yet does not handle async requests - handling everything syncronously.
DatabasePager now by default will enable AsynchronousFileReadHint for http
based file requests
from: DEEP_COPY_STATESETS = 8,
to: DEEP_COPY_STATESETS = 1<<3,
showing clearly that this isn't the _value_ 8, but the _bit_ 8. this is an old pattern i see (and like to promulgate) to make code a bit more readable and maintainable.
"
as not expiring subgraphs quick enough to enable reasonable load balancing.
New version isn't perfect and will need further work, but does at least reduce
the memory footprint by as much as half on test paths on big databases.
The rewritten method no longer uses the the MaximumNumOfRemovedChildPagedLODs
and MinimumNumOfInactivePagedLODs variables so these and associated methods
for accessing them have been removed.
- /** Set the maximum number of PagedLOD child to remove per frame */
- void setMaximumNumOfRemovedChildPagedLODs(unsigned int number) { _maximumNumOfRemovedChildPagedLODs = number; }
-
- /** Get the maximum number of PagedLOD child to remove per frame */
- unsigned int getMaximumNumOfRemovedChildPagedLODs() const { return _maximumNumOfRemovedChildPagedLODs; }
-
- /** Set the minimum number of inactive PagedLOD child to keep */
- void setMinimumNumOfInactivePagedLODs(unsigned int number) { _minimumNumOfInactivePagedLODs = number; }
-
- /** Get the minimum number of inactive PagedLOD child to keep */
- unsigned int getMinimumNumOfInactivePagedLODs() const { return _minimumNumOfInactivePagedLODs; }
in the DatabasePager. The practical effects of these are to greatly reduce startup time
and the time to load an individual scenery tile in FlightGear.
- From my log message:
Minimize the number of StateSets and drawables that are compiled by checking
if they have already been compiled or will be elminated by the
SharedStateManager.
Move the sorting of the dataToCompile queue out of compileGLObjects
into the man pager run function.
Change the SharedStateManager to use maps instead of vectors."
- Get or set the target number of PagedLOD children to remove per frame.
- Get or set the minimum number of inactive PagedLOD to keep.
Corresponding environment variables have been added too.
The default values reproduce the previous DatabasePager behavior."
I've done some additional small modification regarding constness in ReaderWriter and added
mutable on _pluginData so passing data back would be possible too.
Have updated the collada plugin (ReaderWriterDAE.cpp) to use the map to handle options and
have attached the changes.
The stuff in daeReader.h and daeWriter.h are just cosmetic changes to get rid of a warning."
