(http://www.mail-archive.com/osg-users@lists.openscenegraph.org/msg23098.html)
Background: when you access a file over HTTP, you cannot rely on a file extension being present; instead the file's mime-type is conveyed in the HTTP Content-Type response header. This facility adds a mime-type-to-extension map to the registry to handle this.
There are two new osgDB::Registry functions which are pretty self-explanatory:
void addMimeTypeExtensionMapping( mime-type, extension )
ReaderWriter* getReaderWriterForMimeType( mime-type )
I also added the file osgDB/MimeTypes.cpp which houses a hard-coded list of built-in types. I took the list from here (http://www.webmaster-toolkit.com/mime-types.shtml) and then pared it down to include mostly image and video types, editing them to map to existing plugins where possible.
In addition, I updated the CURL plugin to a) install a set of built-in mime-type mappings, and b) use them to look up an extension in the event that the target filename does not have an extension.
Here is a test case. This URL pulls down a JPEG (without a file extension):
osgviewer --image "http://us.maps3.yimg.com/aerial.maps.yimg.com/ximg?v=1.8&s=256&t=a&r=1&x=0&y=0&z=2"
"
Single features are implemented as bits asother enums in ReaderWriter class already does, so that combinations are possible and fast comparison operations are possible
By default all features are enabled.
I have added this virtual method to the dot plugin to get an idea how to use these features.
With this patch osgconv --formats shows an additional line 'features' for each plugin"
"Attached you'll find a proposal for using different
protocols. The idea behind the new code is:
1.) plugins/apps register protocols which they can handle. This is done
via osgDB::Registry::registerProtocol(aProtocolName). Plugins register
supported protocols as usual via ReaderWriter::supportsProtocol(..), the
Registry is updated accordingly.
2.) osgDB::containsServerAddress checks first for an appearance of "://"
in the filename and then checks the protocol against the set of
registered protocols via Registry::isProtocolRegistered(aProtocollName)
3.) the other getServer*-functions changed as well, there's even a
getServerProtocol-function
With these changes filenames/Urls get routed to loaded plugins even with
different protocols than 'http'."
a capping the number of PagedLOD to a sepcified maximum, with pruning of inactive
PagedLOD when the total number of inactive and active PagedLOD goes above the maximum.
To enable the mode set the env var OSG_MAX_PAGEDLOD to a value something like 1000.
New attribute DatabasePager::_expiryFrames sets number of frames a PagedLOD child is kept in memory. The attribute is set with DatabasePager::setExpiryFrames method or OSG_EXPIRY_FRAMES environmental variable.
New attribute PagedLOD::PerRangeData::_
frameNumber contains frame number of last cull traversal.
Children of PagedLOD are expired when time _AND_ number of frames since last cull traversal exceed OSG_EXPIRY_DELAY _AND_ OSG_EXPIRY_FRAMES respectively. By default OSG_EXPIRY_FRAMES = 1 which means that nodes from last cull/rendering
traversal will not be expired even if last cull time exceeds OSG_EXPIRY_DELAY. Setting OSG_EXPIRY_FRAMES = 0 revokes previous behaviour of PagedLOD.
Setting OSG_EXPIRY_FRAMES > 0 fixes problems of children reloading in lazy rendering applications. Required behaviour is achieved by manipulating OSG_EXPIRY_DELAY and OSG_EXPIRY_FRAMES together.
Two interface changes are made:
DatabasePager::updateSceneGraph(double currentFrameTime) is replaced by DatabasePager::updateSceneGraph(const osg::FrameStamp &frameStamp). The previous method is in #if 0 clause in the header file. Robert, decide if You want to include it.
PagedLOD::removeExpiredChildren(double expiryTime, NodeList &removedChildren) is deprecated (warning is printed), when subclassing use PagedLOD::removeExpiredChildren(double expiryTime, int expiryFrame, NodeList &removedChildren) instead. "
multi-threaded paging, where the Pager manages threads of reading local
and http files via seperate threads. This makes it possible to smoothly
browse large databases where parts of the data are locally cached while
others are on a remote server. Previously with this type of dataset
the pager would stall all paging while http requests were being served,
even when parts of the models are still loadable virtue of being in the
local cache.
Also as part of the refactoring the DatabaseRequest are now stored in the
ProxyNode/PagedLOD nodes to facilitate quite updating in the cull traversal,
with the new code avoiding mutex locks and searches. Previous on big
databases the overhead involved in make database requests could accumulate
to a point where it'd cause the cull traversal to break frame. The overhead
now is negligable.
Finally OSG_FILE_CACHE support has been moved from the curl plugin into
the DatabasePager. Eventually this functionality will be moved out into
osgDB for more general usage.
ReaderWriter::ReadResult now has a FILE_REQUEST enum.
ReaderWriter::Options now has a s/getAsynchronousFileReadHint() parameter methods.
libcurl based plugin now detects enabing of the AsynchronousFileReadHint, but
as yet does not handle async requests - handling everything syncronously.
DatabasePager now by default will enable AsynchronousFileReadHint for http
based file requests
from: DEEP_COPY_STATESETS = 8,
to: DEEP_COPY_STATESETS = 1<<3,
showing clearly that this isn't the _value_ 8, but the _bit_ 8. this is an old pattern i see (and like to promulgate) to make code a bit more readable and maintainable.
"
as not expiring subgraphs quick enough to enable reasonable load balancing.
New version isn't perfect and will need further work, but does at least reduce
the memory footprint by as much as half on test paths on big databases.
The rewritten method no longer uses the the MaximumNumOfRemovedChildPagedLODs
and MinimumNumOfInactivePagedLODs variables so these and associated methods
for accessing them have been removed.
- /** Set the maximum number of PagedLOD child to remove per frame */
- void setMaximumNumOfRemovedChildPagedLODs(unsigned int number) { _maximumNumOfRemovedChildPagedLODs = number; }
-
- /** Get the maximum number of PagedLOD child to remove per frame */
- unsigned int getMaximumNumOfRemovedChildPagedLODs() const { return _maximumNumOfRemovedChildPagedLODs; }
-
- /** Set the minimum number of inactive PagedLOD child to keep */
- void setMinimumNumOfInactivePagedLODs(unsigned int number) { _minimumNumOfInactivePagedLODs = number; }
-
- /** Get the minimum number of inactive PagedLOD child to keep */
- unsigned int getMinimumNumOfInactivePagedLODs() const { return _minimumNumOfInactivePagedLODs; }
in the DatabasePager. The practical effects of these are to greatly reduce startup time
and the time to load an individual scenery tile in FlightGear.
- From my log message:
Minimize the number of StateSets and drawables that are compiled by checking
if they have already been compiled or will be elminated by the
SharedStateManager.
Move the sorting of the dataToCompile queue out of compileGLObjects
into the man pager run function.
Change the SharedStateManager to use maps instead of vectors."
- Get or set the target number of PagedLOD children to remove per frame.
- Get or set the minimum number of inactive PagedLOD to keep.
Corresponding environment variables have been added too.
The default values reproduce the previous DatabasePager behavior."
I've done some additional small modification regarding constness in ReaderWriter and added
mutable on _pluginData so passing data back would be possible too.
Have updated the collada plugin (ReaderWriterDAE.cpp) to use the map to handle options and
have attached the changes.
The stuff in daeReader.h and daeWriter.h are just cosmetic changes to get rid of a warning."
Added and cleaned up DeleteHandler calls in osgViewer to help avoid crashes on exit.
Changed DatabasePager across to dynamically checcking osg::getCompileContext(..)
Updated wrappers.
"By repurpose, I'm creating a new plugin that uses much of the .osg fileformat,
but with some changes. Specifically, I'm creating a ".osgfs" plugin, which
represents the scenegraph hierarchy as a filesystem of nested subdirectories and
individual files for each node, rather than nested braces with everything in a
single monolithic file. I intend to incorporate file alteration monitor events
to watch the filesystem for modifications and automatically reload.
The problem I'm running into is osgDB is too tightly coupled to the .osg format.
osgDB::Output::writeObject() contains literal .osg format-specific strings like
"{" to represent the Object hierarchy at too low a semantic level. I propose
using virtual methods; my plugin can then derive from osgDB::Output and
represent Object hiearchy differently.
"
~~~~~~~~~~~~~~~~~~~~~~~~~
This is a simple change to permit databases other than those named
"*.osga" to be used. It is hardcoded in read() at present.
It is non-critical and does not affect existing program functionality.
Registry and Registry.cpp
~~~~~~~~~~~~~~~~~~~~~~~~~
Added a new typedef: typedef std::vector< std::string>
ArchiveExtensionList;
a list of extensions: ArchiveExtensionList _archiveExtList;
and an "add" method: addArchiveExtension(const std::string ext)
This is initialised by adding "osga" in Registry() and used in
Registry::read() where the list is searched for the extension used.
Archive.cpp
~~~~~~~~~~~
This submission is a little more tentative. openArchive() is modified to
automatically add the filename extension to the Registry extension list.
"
To set up extension aliases using a config file, an app calls:
osgDB::Registry::instance()->readPluginAliasConfigurationFile(), passing in the file name as the parameter. (Of course this should be done before loading any files whose names depend on the mapping.) osgDB will search for the file using OSG_FILE_PATH.
The file should contain a line for each mapping, with the "map" extension first, followed by a space or tab, then the plugin identifier. For example, a file containing this line:
flt OpenFlight
would map the ".flt" extension to the OpenFlight plugin."
copy constructors that take a const osg::CopyOp rather than a
osg::CopyOp&, forcing an unnecessary copy. The attached header fixes
this, based off OSG 1.2.
Also fixed duplicate "or" openArchive()'s comment."
Tweaks by Robert Osfield - Moved methods to FileNameUtils, added unix
implementation of getRealPath add extern and OSGDB_EXPORT to function declarations.
default Registry Options object, but it would be useful to be able to
request loading with a different Options object. The attached files
allow you to do that (based off the OSG 1.2 source).
For example, I'm implementing a loader that requires context information
when it pages in subgraphs, which becomes significantly complicated when
multiple scenegraphs are requesting subgraph loads with different
contexts (the loader needs to know which context to use, and the
Registry Options needs to be carefully managed so the context settings
don't clobber each other, especially in multithreaded situations). Being
able to pass an Options instance along with the Node request resolves
this problem."
"My patch is a slight refactoring of the mac specific code in
Registry.cpp and FileUtils.cpp, specifically around the library and
resource file path initilialization methods. This patch cleans up a
lot of the mac specific code by moving repeated code into separate
local functions in FileUtils.cpp that are only compiled on mac builds.
It also adds one function to the API,
appendPlatformSpecificResourceFilePaths in FileUtils. This function
will mirror the already existing
appendPlatformSpecificLibraryFilePaths except for resource file paths.
Currently this function is empty except when built on the mac, in
which case it will add the application bundle's internal Resources
folder and the bundle's parent folder. Previously this code was
implemented as a separate mac specific #ifdef block in Registry.cpp
around the initDataFilePathList method. However, it now is implemented
in appendPlatformSpecificResourceFilePaths in FileUtils.cpp and the
initDataFilePathList method is now the same on all platforms. This
patch should behave the same as before on non-mac platforms.
This patch already includes the fix that Eric mentioned earlier. This
patch is based off of the 0.99 release code. I have tested this patch
using the following testing scheme:
Make a proper bundled application.
While Run from the Finder:
Test that it finds plugins in its internal plugins path.
Test that it finds resources in its internal resources path.
Test that it finds resources in the bundle's parent directory
Test that it finds plugins in the user's Application Support Directory
Test that it finds plugins in the system's Application Support Directory
Test that it finds plugins in the Network Application Support Directory
Check the plugin and resource path lists after they have been
initialized to see if they are in the correct order
While Run from the command line (both from it's parent directory and
from inside the /Contents/MacOS directory) and repeat the above tests.
Check that it also finds plugins and resources within the paths
defined by various environment variables.
Now, Make an application that is NOT bundled/command line only
Test that it does NOT try to look in an internal bundle
plugin/resource directory for plugins or resources.
Test that it finds plugins/resources in the paths defined by the
environment variables.
"
Drawables,StateSet, and osgDB::Registry.
Added cleanup_frame() from to osgProducer::OsgCamerGroup to help with proper
clean of OpenGL objects before exit, and modified osgviewer, osghangglider,
osgwindows examples to do the extra frame call to cleanup_frame() before exit.
much time is allocated to compiling and deleting OpenGL objects, also
added support into osgProducer::OsgSceneHandler.cpp for these new parameters.
The new cotrols are:
DatabasePager::s/getTargetFrameRate(..)
DatabasePager::s/getMinimumTimeAvailableForGLCompileAndDeletePerFrame()
DatabasePager::s/getMaximumNumOfObjectsToCompilePerFrame()
osg::PagedLOD::s/getDatabasePath() and support in PagedLOD::traverse().
osgDB::ReaderWriter::Options::s/getDatabasePath()
osgDB::Input::s/getOptions()
setting of osgDB::Input::setOptions() in ReaderWriterOSG.cpp
src/osgPlugins/ive/DataInputStream::s/getOptions()
setting of src/osgPlugins/ive/DataInputStream::setOptions() in ReaderWriterIVE.cpp
a bool to a enum osgDB::Registru::CacheHintOptions to be used a bit mask flag.
From Robert Osfied, added osgDB::Registry::ReadFileCallback and
osgDB::Registry::WriteFileCallback to allow customizition of read and write
calls.
ammendments by Robert Osfield, adding get/setSharedStateManager()
methods into osgDB::Registry, and clean up fixes in SharedStateManager
for the StateSet arrays.
of iterating through the list of available reader writer, whilst handling
cases where the list itself changes size during iteration through the list.
Previous size changes during iteration was cause invalid iterators, which
in turn was causing a crash.
instead of osgDB::Registry where it original lay. This has been done to allow
fileName strings to be encode data rather than just file names, such as one
requires when using PagedLOD along with plugins for doing dynamic tesselation.
Generalised the osgDB::Field so that its getFloat() method can be used with either doubles or
floats governed by the type passed in - this helps support either float/double
Quat and Matrix classes seemlessly.
this are replaced by the osg::computeLocalToWorld/WorldToLocal() functions
found in osg/Transform.
Made the ReleaseTextureAndDisplayListsVisitor a public nested class of
osgDB::DatabasePager to allow it to be used in the TXP plugin, and added
usage of this visitor to the TXP plugin to make sure that textures and
display lists are released during the update thread.
any reference to these in the distribution across to using unsigned char,
unsigned short etc. This has been done to keep the OSG code more opaque
to what types are.
that they arn't created on the stack inappropriately.
Split the implemention of Matrix up so that it is a simple no referenced counted
class and can be safefly created on the stack. To support referenced counting a
seperate subclass now exists, this is RefMatrix which inherits from both Matrix and
Object.
storage of the path lists into osgDB::Registry, and changed the data
structor from a char* to a std::deque. Changed a names of couple of the
convinience functions in osgDB/FileUtils to better reflect the two
public FilePathList's - DataFilePathList and the LibraryFilePathList.
Added support into the osgDB::Registry::readNode/Image/Object methods
for pushing and popping the path of the current file being loaded.
memory manager published at flipcode.com. This can be turned on
with the OSG_USE_MEMORY_MANGER option which then uses custom global
new and delete operators as well as provide osgNew and osgDelete macro's
which add ability to log line and file from which calls are made.
Updated osg,osgUtil,osgDB,osgText and osgPlugins/osg to use osgNew/osgDelete,
and fixed memory leaks highlighted by the new memory manager.
not delete the object even if its count goes to 0 or below. This should
only be called in special circumstances, the ReaderWriter::ReadResult
being one of them. This new method has allowed the problem of objects
being multiple referenced on return from readNodeFile() & readImageFile().
that used it, all references to the Standard C++ stream classes use the
std::ostream etc convention, all references to "using namespace std" and
"using namespace std::ostream etc" have been removed.
Updates to the osg::Transform, adding preMult and postMult methods and
deprecating the old preRotate,preTranslate,preScale.
Updated the rest of the OSG so that it nolonger uses the deprecated
osg::Transform nodes.
Renamed osgUtil::SceneView::setGlobalState() to
osgUtil::SceneView::setGlobalStateSet() so that the name reflects its
functionality better. Updated osgGLUT::Viewer etc to cope with new
name change.
for settings options in osgDB::Registry, and added the paramter to all of the
reaader/writer plugins. The Options structure by default has an string attached
for packing basic options, however, it also can be subclassed to encapsulate
any users defined option data. In the later case both the client code *and*
the plugin need to be aware of subclass, the plugin will need to use
dynamic_cast<> to assertain its type.
