Table of Contents

  1. Release 2.1b - October 21, 2005
  2. Release 2.0 - January 31, 2005
  3. Release 1.2.1 (Bug Fix Release) - April 19, 2004
  4. Release 1.2 - December 15, 2003
  5. Release 1.1.1 (Bug Fix Release) - September 5, 2003
  6. Release 1.1 (Bug Fix Release) - August 5, 2003
  7. Release 1.0 - May 16, 2003
  8. Release 0.9 - March 7, 2003
  9. Beta 1 - Dec 20, 2002
  10. Alpha 1 - Oct 31, 2002

 

1. Release 2.1b (beta) - October 21, 2005

 

Fedora 2.1b is initially being released as beta, with a final release soon to follow.  This is a very significant release of Fedora since it introduces the new Fedora security architecture (with pluggable authentication and XACML-based policy enforcement), the Fedora Service Framework, and many other new features.  The introduction of the new security architecture resulted in a significant amount of code refactoring, therefore, Fedora 2.1b has been tested with an extensive suite of new JUnit tests.  Some of the system documentation is still evolving.  Most documentation is complete, but several documents are still being enhanced and improved.  To identify documents that will continue to evolve, please look for an icon that says "Draft Doc" under the document title.  As revisions are made, we will publish the most updated versions of these documents on the Fedora web site (www.fedora.info) and send alerts to the Fedora Users mail list.    This release of Fedora was also tested against a large production testbed collection, where bulk loading was performed on for a collection of 1million digital objects with approximately 75M  triples in the Kowari-based Resource Index.   

Also, coinciding with this release is the publication of the Community-Developed Tools page on the Fedora web site (see: www.fedora.info/tools).   This is a clearinghouse for community-developed, open-source applications, tools, and services that work with Fedora repositories.   There are already many tools that are available, so check it out!

 

A. Migrating from Fedora 2.0

The move from Fedora 2.0 to Fedora 2.1b is quite simple and does not require reloading of your repository.  If you are upgrading from Fedora 2.0 to Fedora 2.1b follow the steps below.  More detail on these steps is found in the Fedora Installation Guide and the Upgrade and Migrate Guide.   If you are starting from a release prior to Fedora 2.0, consult the Fedora 2.0 release notes and the Upgrade and Migration Guide for instructions on how to migrate to Fedora 2.0 before upgrading to Fedora 2.1b.

 

NOTE:  Please be aware the base URLs for the Fedora SOAP interfaces have changed in Fedora 2.1b.  The path names for API-A and API-M have changed to facilitate better security management.   The new path names can be seen in the WSDL files for API-A and API-M and in the Fedora Tomcat web.xml file.    In terms of impact on users, this change will only affect developers who have written custom SOAP clients or middleware for Fedora.  If you have written your own custom SOAP connector for Fedora,  be sure to reference the new path names within the SOAP base URLs:

 

B. New Features

 

  1. New Open-Source License (ECL)
    For version 2.1 we have decided to move from Mozilla Public License 1.1 (MPL) to the Educational Community License 1.0 (ECL). The main difference between the two licenses is that the ECL imposes no compulsion to contribute changes to source code. Our experience has been that consumers of our code, both academic and commercial, see it as in their interest to contribute code back anyway. In several cases, this requirement has been a barrier for adoption of Fedora.   Note that this license makes the software openly usable by anyone for any purpose. The only requirement imposed is that anyone using our source code, making changes and distributing the changed source, is required to make it clear that the new distribution is different and to describe the changes made.  Also note that this change only applies to version 2.1. All earlier versions continue to be covered by their original license.
     
  2. New Security Architecture ? Four Security Configuration Options for a Repository
    Fedora 2.1b introduces a configurable security architecture, that provides options for SSL and authentication at the Fedora web service API level.   There are four base security configurations provided with the Fedora 2.1b server distribution: (1) "ssl-authenticate-apim" configuration which uses SSL and enables authentication for API-M, (2) "ssl-authenticate-all" configuration which uses SSL and enables authentication for both API-M and API-A, (3) "no-ssl-authenticate-apim" configuration which does not use SSL but still enables authentication for API-M, and (4) "no-ssl-authenticate-all" configuration which does not use SSL, but still enables authentication for API-M and API-A. Unlike Fedora 2.0, there is no longer a "default" configuration. Before starting the Fedora server, you must run the new server setup utility (fedora-setup) to select a base security configuration that will initialize the servers security configuration.   After setup, you can customize that base configuration as needed.   The document entitled "Securing Your Fedora Repository" describes the server setup process and the nature of the different base security configurations.
     
  3. New Security Architecture ? Authentication
    The Fedora repository server is built on top of Apache's Tomcat, and authentication is channeled through the normal Tomcat mechanisms (e.g., HTTP Basic Authentication, HTTPS Client Authentication).  When you select one of Fedora's four security configurations, an appropriate rendition of the Tomcat configuration file (web.xml) will be copied into the Fedora Tomcat configuration directory.  This configuration file will set up authentication for Fedora web services (via <login-config>  and <security-constraint> elements).  Under the hood, Tomcat comes with different Realms, where a realm is defined as a "database" of usernames and passwords that identify valid users of a web application (or set of web applications), plus an enumeration of the list of roles associated with each valid user  [see: Apache Tomcat Realms].      Tomcat defines a Java interface (org.apache.catalina.Realm) that can be implemented by "plug in" components to "connect" a servlet container to some existing authentication database.   Fedora 2.1b comes pre-configured with a customized JAAS Realm whose purpose is to multiplex several authentication sources.   By default, Fedora's JAAS multiplexing Realm in configured to deal with a simple user/password file.  There is also support for acquiring user attributes from an LDAP source.  Authentication methods other than the two described here are possible through Tomcat, but require configuring other Realm or Login Modules obtained separately from the Apache Project or third parties, or writing your own custom Realms or LoginModules.  Additional user attributes can be merged with those which Tomcat provides by configuring a site-written servlet filter, which calls the Fedora API.   For more details refer to the documents entitled "Securing Your Fedora Repository " and "Fedora Security Architecture - Authentication."

     
  4. New Security Architecture - XACML Policy Enforcement
    A major feature of the new Fedora security architecture is the introduction of the eXtensible Access Control Markup Language (XACML) and an XACML-based policy enforcement module.    Developed by the OASIS Consortium, XACML is an XML-based markup language to encode access control policies.  The policy language is flexible and enables the specification of fine-grained, machine-readable policies that can be used to control access to Fedora web services, Fedora digital objects, datastreams, disseminations, and more.    Since a policy is only worth its salt if it can be enforced, Fedora 2.1b introduces a new Authorization module implemented as part of the core Fedora repository service.    The Authorization module is built upon the Sun XACML engine.  Each XACML policy defines: (1) a "target" describes what the policy applies to (by referring to attributes of users, operations, objects, datastreams, dates, and more), and (2) one or more "rules" to permit or deny access.  Version 2.1b provides a Fedora-specific policy vocabulary for referring Fedora operations and Fedora-specific entities within in XACML policies.   For a description of how the policy enforcment module is implemented, see the document entitled "Fedora Authorization with XACML Policy Enforcement."  Regarding policies, Fedora supports both repository-wide policies (that specify broad access controls that apply to the entire repository), and object-specific policies (that specify rules for a single object, and can even be stored within the object in a special datastream).  Fedora 2.1 comes out-of-the-box with a set of default repository-wide policies that establish baseline access controls that are equivalent to what was provided in Fedora 2.0.   These default policies restrict access to the Fedora management web service (API-M) to only the Fedora administrator, permit open access to the Fedora access web service (API-A), and establish some other basic controls (e.g., allow access to API-M only from localhost; restrict policy management to administrator).   In addition to the default policies (which can be modified), any number of custom XACML policies can be written and loaded into Fedora.   For assistance in creating new policies for your repository and for your objects, see the "Fedora XACML Policy Writing Guide."
     
  5. Fedora Policy Reload Utility
     The Fedora Policy Reload Utility ("fedora-reload-policies" on the command line) provides an easy way to load new or modified policies into Fedora without having to restart the Fedora server.   With the addition of the XACML-based Policy Enforcement module, there is the opportunity to modify existing policies or add new policies over time.  All "active" policies must be stored in the active policies storage locations configured for the Authorization module in the Fedora server configuration file (fedora.fcfg).   Once policies are copied into one of these "active" policy directories, the Fedora server can locate them and load them into memory for policy enforcement.   Every time the Fedora server is started policies are loaded from the active policy directories.    It should be noted that Fedora supports both repository-wide policies and object specific policies.  The Policy Reload Utility will load only repository-wide policies.  For performance benefits, object-specific policies are loaded on demand when the repository receives a request for that particular object.
     
  6. Fedora Policy Validation Utility
     To support authoring and maintenance of XACML policyies for Fedora, the policy validation utility (i.e., "validate-policy  {policyfile name}"on the command line) is provided with Fedora 2.1b.  This utility will validate an XACML policy against the XML schema defined by Sun for XACML.  It is recommended that this utility be invoked on all policies before they are moved to the active policies storage location to prevent errors when the Fedora server attempts to load XACML policies for use by the Fedora Authorization module.
     
  7. New Security Architecture ? Backend Security for Service call-backs
    Backend security is applicable to repositories that contain digital object Disseminators that call upon external web services - and where a high level of security is desired.   Repositories that do not use such external dissemination services do not require any custom configuration of this new security feature.       The term "backend services" is used to describe the set of web services that are used by Fedora Disseminators.   Disseminators are optional components in the Fedora digital object model that allow for service-based views of the digital object.  A Disseminator points to a "Behavior Mechanism" which contains a WSDL service description that is used by the Fedora repository service to call upon an external supporting service at runtime.   These external services that Fedora calls upon are referred to as "backend services."   When Fedora makes a call to a backend service, it will typically send a reference to a datastream (stored in a digital object in the repository) that Fedora wants the external service to process in some way (e.g., a transformation of the datastream content).   When a backend service receives such a request from Fedora, it must call back to Fedora to get the datastream content that it is required to process.   This is where backend security comes in.  If the Fedora repository is set up in a secure manner (SSL and authentication), a backend service will be treated like any other incoming client to Fedora.  The service must make calls to Fedora in a secure manner, and it may need to provide credentials to Fedora.   To support the secure interaction of backend services with a secure Fedora repository, Fedora 2.1 provides a the "beSecurity Configuration Tool" for registering each backend service and specifying how it should call back to Fedora.  For more information on configuring backend security refer to the document entitled "Backend Services Security Configuration."
     
  8. Fedora Service Framework
    As of Fedora 2.1b, the Fedora Service Framework is being introduced to facilitate the integration of new services with the Fedora repository.   The framework takes a service-oriented architecture approach to building new functionality around a Fedora repository.  While the Fedora repository, itself, exposes its functionality as a set of web service interfaces, all of these interfaces belong to the Fedora web application that runs in its own Tomcat   The new Fedora Service Framework allows new services to be built around the core repository - as stand-alone web applications that run independently of the Fedora repository.   While Fedora repository functionality can still be extended with new modules, the intent is to keep the repository service focused on the core functions of a repository.   Yet, there are many other services that are beneficial companions to a repository, such as specialized ingest services, workflow services, preservation services, and many others.   These are the kinds of services that the framework is intended to support.   There are two main benefits to the service framework approach:  (1) it allows new functionality to be added as atomic, modular services that can interact with Fedora repositories, yet not be part of the repository, (2) it makes co-development of new services for Fedora easier since each service can be independently developed and plugged  into the framework.   The Fedora development team has developed an initial set of services (Directory Ingest and PROAI described below), and will continue to develop new services over the coarse of Fedora Phase 2 (205-2007), especially services for workflow, preservation, and search.   New services will be part of the main Fedora distribution and will be kept up to date with new versions of the core Fedora repository distribution.   Members of the Fedora community will be collaborating on the development of services and will contributed back to the Fedora Project.    Further documentation will be provided to establish guidelines on how services should be designed to effectively plug into the framework.  In the mean time, developers of new services can follow the design patterns of the Directory Ingest and PROAI services.
     
  9. New Directory Ingest Service (DirIngest)
    New as part of the Fedora Service Framework, is the new Directory Ingest Service, which is a stand-alone web service application that interacts with a Fedora repository to provide for ingesting directories or archives or files into a Fedora repository.  The service receives a zip/jar file of files in directory hierarchy (i.e., a file archive), including a METS-based manifest file.  The Directory Ingest Service will receive the archive, open it up, create a digital object in the repository for every file and directory, and populate the RELS-EXT datastream preserve the relationships between files and directories as represented in the input archive.
     
  10. New OAI-Provider Service (PROAI)
    New as part of the Fedora Service Framework, is the new Fedora OAI Provider Service, which is a stand-alone web service application that is a highly configurable OAI provider.   The PROAI service can be set up to harvest any type of datastream or dissemination of digital objects in a Fedora repository.  It also supports OAI sets.  (Note: PROAI can be used outside the context of Fedora too, by writing a custom adapter.)  Note also that the simple OAI provider interface on the core Fedora repository is still available.   The new OAI service provides much more functionality and better performance.
     
  11. Resource Index
     The Resource Index has undergone significant testing and a number of bugs have been fixed (see Bug Fixes below).  Most notably, a memory leak was discovered in the Kowari triplestore software that underlies the Resource Index that was the cause of reported Resource Index corruption problems.  (It should be noted that corrupted Resource Index can be restored using the Fedora Rebuilder Utility that was pre-release to those having the problem in Fedora 2.0.  The Rebuilder is now part of Fedora 2.1b.)   There have been minor changes to the Resource Index data model.  This should be transparent, unless you have written applications or middleware with static queries to the Resource Index, in which case you may have to make modifications to those queries.  See the Resource Index Guide for more information.  Also with Fedora 2.1b, a new indexing level is introduced for the resource index.  This new level (Level 3), will index parameterized disseminations as concrete "representations" of digital objects, whenever there is a fixed domain of values for method parameters.  This is explained in detail in the Resource Index Guide.  Also, as explained in below, the RELS-EXT datastream can be used more freely now, beyond just the expression of object-to-object relationships, making the Resource Index more useful for asserting community-specific properties about objects via RDF.
     
  12. Relaxation of REL-EXT datastream constraints
    In Fedora 2.0, the Relationships datastream in an object (identified as "RELS-EXT") was reserved for assertions of object-to-object relationships.   Now, in Fedora 2.1b, the RELS-EXT datastream can be used as a means of creating extensible properties for digital objects (i.e., properties not defined in the FOXML schema).  The benefit of putting such properties in the RELS-EXT datastream is that the are automatically indexed in the Resource Index, and thus are searchable via the RISearch interface.  Previously, the RELS-EXT datastream was validated by Fedora to ensure that every assertion was of the form  "PID -- relationship -- anotherPID"  where the subject was the PID of the object containing the assertion, the predicate a relationship, and the object of the predicate was the PID of another Fedora object.   New in Fedora 2.1b is that validation of the RELS-EXT datastream has been relaxed so that it can be used to assert other kinds of custom properties about the digital object.    Specifically, the object of the predicate of an assertion does NOT have to be another Fedora PID;  it can be any URI or literal value.   Thus RELS-EXT can be used to associate properties defined in any ontology with a Fedora digital object.  The example below shows the content of a RELS-EXT datastream with three assertions about the Fedora object "demo:99."   The assertions are: (1) a object-to-object relationship defined in the Fedora ontology namespace that says "demo:99" is member of the collection object identified by "demo:c1"  (2) a relationship (myns:isPartOf) defined in custom namespace that says "demo:99" is part of a non-Fedora entity identified by "doi:10.123/456" and (3) an object property (myns:owner) defined in custom namespace that has a value of  "Jane Doe."    The second and third assertions are examples of what is newly supported in Fedora 2.1b that was not previously supported.   The second assertion is in the form "PID -- someRelationshipName -- someURI" and the third assertion is in the form of  "PID -- somePropertyName -- someLiteralValue."

     
    <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" 
 xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
 xmlns:fedora="info:fedora/fedora-system:def/relations-external#"
 xmlns:myns="http://www.nsdl.org/ontologies/relationships#"> 

 <rdf:Description rdf:about="info:fedora/demo:99">
    <fedora:isMemberOfCollection rdf:resource="info:fedora/demo:c1"/>
    <myns:isPartOf rdf:resource="doi:10.123/456"/>
    <myns:owner>Jane Doe<myns:owner/>
 </rdf:Description>

</rdf:RDF>


     

  13. Rebuild Repository Utility
    The Fedora Rebuild utility is an interactive utility that can be used when the Fedora repository is somehow corrupted.  The utility rebuilds the repository by crawling the repository storage directories where the FOXML digital objects reside.   The utility can rebuild the SQL database (registries, search tables, and other tables), the Resource Index, or both.    This is useful for a number of purposes, such as migrating from Fedora 2.0 to Fedora 2.1b, and also if a database or index becomes corrupted.   Additionally, if the Fedora administrator want to change from using the McKoi database to using the MySQL database, (or from MySQL to McKoi) the existing database can be purged, the fedora configuration file (fedora.fcfg) can be edited, and the fedora-rebuild utility can then be run to correctly populate the tables in the new database system.   Similarly, the fedora Rebuild Utility can also be useful for rebuilding the Kowari triplestore of the Resource-Index if a problem or corruption occurs.
     
  14. Command-line utility syntax changes
    With the introduction of the new security architecture for Fedora, the repository can be configured for SSL or non-SSL.  Therefore, the syntax for Fedora client command line utilities has been slightly changed to accomodate the a parameter for the protocol that will be used when calling Fedora.  Refer the the Client Command Line Utilities Guide for the new syntax for each command line utiltiy.
     
  15. Batch Utility Enhancements:  
    You can now use either a FOXML or METS template file when building objects with the Fedora batch utility.  The batch utility now detects the format of the template file and then generates objects based on the format of the template file so you can create either METS or FOXML  objects now. 
     
  16. Improved logging using log4j
    Fedora server logging has been standardized to use log4j.   In addition to the general repository server log (created each time the Fedora server is started and named with a date-time stamp), there are now separate logs for Resource Index activity (ri.log) and for transactions sent to the Kowari triplestore that underlies the Resource Index (see: trippi.log and replay.log).   With the log4j logging in place, the next release of Fedora will include a refactoring of log messages to eliminate excessive log messaging, and to better categorize the severity levels of existing log postings. 
     
  17. FedoraClient utility class for building new clients
     
  18. Performance and Scale testing
    Bulk loading was performed on the NSDL repository, which provided a testbed of 1 million digital objects with approximately 75M  triples in the Kowari-based Resource Index.  These tests were run on a 64 bit architecture due to heavy use of the Resource Index and Kowari memory requirements.  These tests were run by NSDL middleware that heavily exercised the repository in terms of adding/modifying/deleting large numbers of objects.   The results of these tests will be published soon on www.fedora.info. 
     
  19. New Tools Page for Software Contributions by Community
    A new page now exists on the Fedora web site (www.fedora.info) for downloading community-developed, open-source tools, applications, services that work with Fedora.  Check it out!
     

 

C.  Bug Fixes

        Refer to Fedora's Bugzilla for full descriptions of the resolved bugs.

 

  1. Bugzilla #68:  Inconsistent treatment of null vs. empty space on API-M method parameter values.  
    Wthin the operations modifyObject, modifyDatastream, and modifyDisseminator on API-M null parameter values were not handled consistently.  Now a null parameter value means that a change will be made to the object/datastream/disseminator property that the null parameter value is targeting.   An empty string parameter value means that property will be changed to an empty string.  Any non-empty string parameter value means that property will be changed to the specified parameter value.
     
  2. Bugzilla #87-88  Minor display issues on Fedora web site.  These are fixed on www.fedora.info.
     
  3. Bugzilla #89: Classpath error when running fedora-stop.bat.  
    This was traced to servlet-api.jar not being found on the classpath.  Specifically, TC_COMMON is referenced at line 11 but not defined till line 15.  The fix was to move line 15 of  fedora-stop.bat to be after line 8 and before line 11.
     
  4. Bugzilla #93:  Fedora Tutorial has broken dissemination example.  
    In Fedora Tutorial 2 Example 5 there is an example image object (demo:800) with disseminators using the BDef demo:600.  The greyscale and and resize disseminations are now fixed.
     
  5. Bugzilla #97-98:  Command line scripts break if JAVA_HOME has spaces.  This problem was reported on fedora-stop.bat and fedora-ingest-demos.  This has been fixed.
     
  6. Bugzilla #99:  The API-A listDatastreams operation via SOAP returning incomplete list.  
    When running listDatastreams via API-A (SOAP), the list is incomplete, but when running against API-A-LITE (REST), it works fine.  The incomplete list consists only of datastreams that are used by disseminators. cause was getting a "cached" context in FedoraAPIASOAPHTTPImpl.java.  The fix was to use a non-cached context so the full, acurate list of datastreams is returned.
     
  7. Bugzilla #100:  Fedora debug script required upgrading.  The fedora-debug.bat was not previously updated to be compliant with Fedora 2.0, which has now been fixed.
     
  8. Bugzilla #101 and 104: MySQL scripts fail when MySQL path includes spaces.  
    The scripts affected were mysql-config.bat  and mysql-drop.bat.  The fix is that If the MySQL path contained spaces, enclose the path in double quotes and the script will handle the path correctly (e.g., mysql-config.bat "C:Program files\MySQL\MySQL server 4.1").
     
  9. Bugzilla #105-106:  The MySQL script "mysql-drop-db.bat" does not work with MySQL 4.1. 
    This was a bug in mysql-drop-db.bat and has been fixed in cvs as of 2/18/2005. The optional mysql41_flag argument has been added to the script. When specified, the script will set the default character set to utf8 and collation to utf8_bin for the fedora database when it is recreated. This script is primarily a convenience utility when testing and you need to drop and recreate the Fedora SQL database.  Refer You can accomplish the same tasks manually using the MySQL commands.  Refer to Bugzilla for more details on how to do this. 
     
  10. Bugzilla #107:  Use of "distinct" ignored in Resource Index Search. 
    When using the Resource Index REST-based search interface (risearch), the use of "distinct" didn't work in queries.  The cause was that risearch was ignoring the "on" value.  The servlet that implements risearch was looking for a value of 'true' instead of a value of "on."  This caused "distinct" to not work.  The fix was done within the trippi libraries -- now it accepts 'on' or 'true' for the value.  A replacement trippi library (trippi.jar) was created and checking into cvs.
     
  11. Bugzilla #108:  Bad link on Download Page for Fedora 1.2.1.  
    The download page for 1.2.1 has a bad hyperlink to the patch download. Also the link "Download" on this page pointed to an old location.  Both are fixed.
     
  12. Bugzilla #110:  Resource Index Corruption.  
    After the introduction of the Resource Index in Fedora 2.0, there were reports of out of memory errors occurring related to the Kowari triplestore.  This often led to corruption of Resource Index.  This was     traced to a memory link in Kowari which was caused by a failure to properly clean up unused FreeList$Phase objects in one of the FreeList constructors.  The problem showed up when doing many inserts and deletions. More information can be found at:  http://prototypo.blogspot.com/2005/09/kowari-memory-leak-found-and-fixed.html
     
  13. Bugzilla #114: Resource Index had missing lastModifiedDate property on datastream disseminations nodes.  
    Resource Index datastream disseminations had createdDate instead of lastModifiedDate properties.  According the the Resource Index data model, Disseminations should *not* have a createdDate property.  Now they have lastModifiedDates as expected.
     
  14. Bugzilla #116: Possible SQL database corruption during ingest when failure occurs.
    When ingesting an object and a failure occurs after the object is initially registered, the registry entries in the sql db table doRegistry are not properly backed out. The result is that PID entries are left in the doRegistry table so that subsequent attempts to re-ingest will fail with the error being that the PID already exists.  bug had to do with Exception handling during the ingest sequence. The bug was introduced with code changes related to Fedora 2.0. The occurence of this bug in the 2.0 release should be rare but could occur if a fatal error occurred with the sql database transactions during the ingest sequence after the object had been registered. The symptom would be a failed ingest and then the inability to re-ingest the same object after the failure without manually removing the PID entry from the doRegistry table.Exception handling has been modified in DOManager and DOWriter to account for this scenario and properly backout any entries in the doRegistry table. Exceptionon handling has been modified in DOManager and DOWriter to account for this scenario and properly backout any entries in the doRegistry table following a failure during the ingest process.
     
  15. Bugzilla #117: Inline datastream xmlns placement problem.
    An inline xml datastream with valid, namespaced XML.  When tried to export the object, it failed (but purging it worked).The problem seems to show itself when you have multiple 'xmlns=' ("default"namespace) declarations in one inline xml chunk.What happens is, somewhere along the line the default namespace declarations getmoved to the root.  And this makes for invalid (not well-formed) xml (repeatingattributes of the same name are not allowed in the same element).This is fixed in FOXMLDODeserializer and METSLikeDODeserializer.  The fix was tokeep a map of all prefix-to-uris and a list of prefixes that needed to be written while parsing each element inside xml content.
     
  16. Bugzilla #125: Fedora Administrator does not allow editing of "application/xml" MIME-typed datastreams.  This is now fixed.
     

 

D.  Known Issues

  1. If you are using Disseminators that bind to external services AND if you configure Fedora to authenticate the API-A service, then you should NOT set the Fedora server hostname to LOCALHOST since this is not a resolvable domain name.  Disseminators with external services require that external services be able to call back to the Fedora repository to obtain datastreams.   When an external web service tries to call back to the Fedora, repository,  there must be a publicly registered domain name for the Fedora server or else the external service will not be able to contact the repository.   To avoid this problem, change the Fedora server hostname in the Fedora server configuration file (fedora.fcfg) to a registered domain name.  Note that if you are running as LOCALHOST when API-A authentication is ON, you will encounter exceptions in demo objects disseminations that depend on external services (demo:6, demo:7, demo:10, demo:11, demo:17).
     
  2. When the Resource Index is configured to be ON,  care should be taken to estimate the number of triples per object that will be indexed in the Resource Index (in the Kowari triplestore in particular).  Consult the document named "Triples in the Resource Index" for an understanding what is indexed in the Resource Index, and to estimate the number of triples in that will be inserted into the Kowari triplestore.   For very large triplestores, a 64 bit architecture is required for optimal performance and memory management.  Testing of Kowari has shown that performance on a 32 bit architecture degrades when the Kowari triplestore get to approximately 50M triples, but that with a 64 bit architecture, Kowari scales well to 250M triples (see: http://www.itee.uq.edu.au/~dwood/docs/www2005-kowari.pdf).
     
  3. Also, when bulk loading a repository that is estimated to have a very large Resource Index (i.e., triplestore > 50M triples), it is recommended that the Fedora server be restarted periodically to optimize performance.  Bulk load tests on Fedora 2.1b indicate that there may be a slow memory leak when doing large numbers of delete/insert sequences on the Kowari triplestore.    This potential issue is currently under investigation.  As a cautionary measure, periodic stopping and restarting of the Fedora repository server will reclaim memory and prevent the possibility of an out-of-memory exception (which might have unpredictable effects on the Kowari triplestore). 
     
  4. Under certain situations, the HTTP error response generated by a dissemination request can mask the true underlying error. For example, consider a backend service that attempts to get a datastream request from Fedora and Fedora responds with a HTTP error response of 403 indicating authorization with the Fedora server was denied. If the backend service does not properly handle this response and just returns ServerException(500 error) for any non-200 response, then the originating dissemination request will only "see" the 500 error returned by the backend service and not the underlying 403.
     
  5. Running the server in "debug" mode will slow down performance.  You can turn debugging on/off in the Fedora server configuration file (fedora.fcfg).
     
  6. Modification of Behavior Definition and Behavior Mechanism objects not yet supported, but will be made available in the next release of Fedora.
     
  7. One early adopter of Fedora 2.1b reported anomalous 401 status codes returned after a period of normal server use.  To date, this observation hasn't been replicated, and the cause is unknown.  The problem was fixed by changing from option B to option C as described in the document titled "Securing Your Fedora Repository", section 4.
     
  8. The exception fedora.server.errors.servletExceptionExtensions.Ok200Exception, (and associated stacktrace) is produced by fedora.server.ServerController and can safely be ignored.
     

 

 

2.  Release 2.0 - January 31, 2005

Fedora 2.0 is a major release that culminates Phase I of the initial Fedora project that began in October 2001 and ended in October 2004. Fedora 2.0 includes significant new features and improvements including the introduction of the Fedora Object XML (FOXML) schema as the new internal storage format for objects, introduction of the Resource Index that provides enhanced search capability, introduction of a Batch Modify utility, upgrades of all third party libraries, performance enhancements, and a number of bug fixes.

IMPORTANT!! - Migrating From Fedora Release 1.2.1

Users migrating from Fedora Release 1.2.1 need to follow the steps outlined in the section entitled, Migrating From Fedora 1.2.1 to Fedora 2.0, and apply patch number 2 to their existing Fedora 1.2.1 repository BEFORE attempting migration to Fedora 2.0. Failure to do so can severely limit future migration options.

If you are running a version of the Fedora software prior to version 1.2.1, you will need to migrate your server software up to version 1.2.1 first, and then proceed with the migration from Fedora 1.2.1 to Fedora 2.0. Steps for migrating to Fedora 1.2.1 can be found in the release notes section for Fedora 1.2.1.

A. Migrating From Fedora 1.2.1 to Fedora 2.0

If you are upgrading from Fedora 1.2.1 to Fedora 2.0 follow these steps:

  1. Download patch number 2 from http://www.fedora.info/download/1.2.1/patch2.
  2. Make sure you are running v1.2.1 of the Fedora server software.

    DO NOT APPLY THIS PATCH TO ANY OTHER VERSION OF FEDORA!

    To check your repository's version, check the "describe" page. For instance, if you're running Fedora on localhost:8080, visit http://localhost:8080/fedora/describe

  3. Shut down your Fedora 1.2.1 server (using fedora-stop)
  4. Make a backup of the METSLikeExportDOSerializer.class file.

    Under Windows, make a backup of the following file:

    %FEDORA_HOME%\server\tomcat41\webapps\fedora\WEB-INF\classes\fedora\server\storage\translation\METSLikeExportDOSerializer.class

    Under Unix, make a backup of the following file:

    $FEDORA_HOME/server/tomcat41/webapps/fedora/WEB-INF/classes/fedora/server/storage/translation/METSLikeExportDOSerializer.class
  5. Copy the METSLikeExportDOSerializer.class from this patch into the above directory replacing the one there.

    Under Windows, copy the METSLikeExportDOSerializer.class from patch into the following directory

    %FEDORA_HOME%\server\tomcat41\webapps\fedora\WEB-INF\classes\fedora\server\storage\translation

    OVER THE EXISTING FILE.

    Under Unix, copy the METSLikeExportDOSerializer.class from patch into the following directory

    $FEDORA_HOME/server/tomcat41/webapps/fedora/WEB-INF/classes/fedora/server/storage/translation

    OVER THE EXISTING FILE.

  6. (OPTIONAL) If you installed the source code distribution of Fedora, you should also copy the included METSLikeExportDOSerializer.java file into your source directory.

    Under Windows, copy the METSLikeExportDOSerializer.java file from the patch into the following directory

    %FEDORA_HOME%\src\java\fedora\server\storage\translation

    OVER THE EXISTING FILE.

    Under Unix, copy the METSLikeExportDOSerializer.java file from the patch into the following directory

    $FEDORA_HOME/src/java/fedora/server/storage/translation

    OVER THE EXISTING FILE.

  7. Re-start your Fedora 1.2.1 server (using fedora-start). Your Fedora 1.2.1 repository is now ready for exporting its objects to the new server.
  8. Download the Fedora 2.0 software from http://www.fedora.info/download
  9. Install and configure the new Fedora 2.0 repository with a different host and port than the Fedora 1.2.1 server.
  10. Start the Fedora 2.0 server (using fedora-start).
  11. There are two ways to ingest objects from the 1.2.1 server: 1) use the admin GUI client to ingest the objects from the 1.2.1 repository or 2) use the fedora-ingest command-line utility to ingest the objects from the 1.2.1 repository
    1. Using the admin GUI client to ingest objects from the 1.2.1 repository
      1. Start the fedora admin GUI client (using fedora-admin).
      2. Under the File menu item, select File, Ingest, Object By Type, From Repository.
      3. Specify the hostname and port of the Fedora 1.2.1 server from which you want to ingest objects. Specify the administrator username and password for the 1.2.1 repository.
      4. Select all three object types including Behavior Definition, Behavior Mechanism, and Data object and click OK.
      5. The Fedora 2.0 server will now ingest all objects from the 1.2.1 repository.
      OR
    2. Using the fedora-ingest command-line utility to ingest objects from the 1.2.1 repository.
      1. Run the fedora-ingest command-line utility which is found in the client/bin directory.

        Under Windows, this file is found in:

        %FEDORA_HOME%\client\bin\fedora-ingest.bat

        Under Unix, this file is found in:

        $FEDORA_HOME/client/bin/fedora-ingest.sh
      2. Arguments for the fedora-ingest utility should include the following:
        • Type of ingest: specify R for repository
        • Hostname and port of the 1.2.1 repository from which you wish to ingest objects: source-host:source-port
        • Administrator username of the 1.2.1 repository: fedoraAdmin
        • Password for the administrator username of the 1.2.1 repository: fedoraAdmin is the default
        • Type of objects to ingest: specify DMO for all three object types.
        • Hostname and port of the 2.0 repository into which you are ingesting the objects: destination-host:destination-port
        • Administrator username of the 2.0 repository: fedoraAdmin
        • Password for the administrator username of the 2.0 repository: fedoraAdmin is the default
        • Log message: use "" for no message.

          For example:

          fedora-ingest r src-host:src-port fedoraAdmin fedoraAdmin DMO dest-host:dest-port fedoraAdmin fedoraAdmin ""
  12. After successfully ingesting all objects from the 1.2.1 repository, shutdown the 1.2.1 repository (using fedora-stop).
  13. If you desire to run the Fedora 2.0 repository under the same hostname and port as the 1.2.1 repository, you can now shutdown the 2.0 repository using fedora-stop, reconfigure the host and port in the fedora.fcfg file for the 2.0 repository to match those of the previous 1.2.1 repository, and restart the Fedora 2.0 repository.

B. New Features

  1. Fedora Object XML (FOXML)

    A new XML schema has been developed that provides a simple and unambiguous expression of the Fedora digital object model. Previously, Fedora digital objects could only be expressed in XML using an extension of the METS schema. FOXML is the new internal storage format for digital objects inside a Fedora repository. However, Fedora supports both METS and FOXML as ingest and export formats (see #2 below). The introduction of FOXML was motivated by several requirements: (1) simplicity - user feedback called for a conceptually easy mapping of the Fedora concepts to an XML format. Users wanted an obvious sense of how to create Fedora ingest files, especially those who are not familiar with formats such as METS and DIDL, (2) optimization and performance - the FOXML schema was designed to improve repository performance, both at ingest and during disseminations. Overall ingest performance has been positively affected with FOXML, especially in the validation phases, (3) flexibility - internal storage of FOXML enables easier evolution of functionality in the Fedora repository, without relying on Fedora-specific extensions to the METS schema. FOXML also introduces several new features to the Fedora Object Model including, the ability to store alternate identifiers for datastreams, and the ability to store format URIs for datastreams and objects (i.e., in anticipation of services such as the Global Digital Format Registry (GDRF). Refer to the user documentation for more details about FOXML and new features of the Fedora Object Model.

    The XML schema for FOXML is available at:

    http://www.fedora.info/definitions/1/0/foxml1-0.xsd

  2. Ingest and Export with Different XML Formats

    The Fedora project is committed to providing repository ingest and export functions that are compatible with community developed standards for digital object XML encoding. This is motivated by both interoperability and digital preservation concerns, where flexibility in the transmission format of digital objects is an important feature. In support of this, the ingest and export operations of API-M have been redesigned to allow submission and retrieval of digital objects in several alternative XML formats. In release 2.0, digital objects can be ingested and exported in either METS (Fedora extension) or FOXML. In future releases, we will add support for METS 1.3 and MPEG21/DIDL. Internally, the Fedora repository system provides a translation module that is responsible for mapping digital objects to the different supported formats. See the new "Ingest and Export Guide" in the system documentation for more details.

  3. Relationship Metadata for Digital Objects

    Fedora now supports the assertion of object-to-object relationships. Fedora digital objects can be related to other Fedora objects in many ways. For example there may be a Fedora object that represents a collection and other objects that are members of that collection. Also, it may be the case that one object is considered a part of another object, a derivation of another object, a description of another object, or even equivalent to another object. Relationship metadata of this sort can now be encoded in Fedora digital objects in a special reserved datastream (identified as "RELS-EXT"). Relationships are expressed as RDF/XML. Fedora provides a default relationship ontology that defines a set of common relationships (see: http://www.fedora.info/definitions/1/0/fedora-relsext-ontology.rdfs). Also, there is support for community or user-defined relationships. The relationships datastream is automatically indexed by Fedora as part of the new Resource Index module (described below), which provides RDF-based searching of the repository, including object-to-object relationships.

  4. Resource Index

    The Resource Index Module provides an RDF storage and query service for object metadata and relationships in Fedora. By default, the Resource Index indexes object properties, datastream, and dissemination information. In addition, user-defined, inter-object relationships expressed the RELS-EXT datastream are also automatically indexed. Examples of relationships between digital objects that may be expressed in the Resource Index include well-known management relationships such as the part-whole links between individual chapters and a book, and semantic relationships useful in digital library organization such as those expressed within the Functional Requirements for Bibliographic Records (FRBR). The query interface is exposed as a web service, providing the foundation for external services. Refer to the user documentation for additional information about the Resource Index.

  5. Batch Modify Utility

    A new utility has been added to the Fedora administrative client that allows repository administrators to perform object modifications in batch mode. The utility is built using the management API method of API-M so that most of the API-M methods can be run against groups of objects. The utility works by creating an xml-encoded input file of modify directives. This directives files is then processed in sequence to carry out the designated tasks. A sample batch modify directives file is included in the distribution in the batch-demo directory. Refer to the Batch Modify Utility user documentation for additional information regarding the Batch Modify Utility.

  6. Repository Administrator Reporting Tools

    A new Reporting interface has been added for obtaining simple administrative reports about objects in the repository. The interface consists of several static queries that can be performed on the repository to obtain statistics about objects. These queries currently include object counts based on type of object, creation date, and pid namespace. Refer to the user documentation for additional information.

  7. Example Client for Making SOAP calls to Fedora APIs

    A sample java client application is provided that demonstrates how to make SOAP calls to API-M and API-A. This client is written as a single java class that connects to a Fedora repository, and makes a set of successive set of SOAP calls to ingest an object from a FOXML XML file, then add datastreams, modify datastreams, export the object, and several other useful examples. It is intended as a quick and easy starting point for Java programmers who want to write their own clients to the Fedora SOAP interface. The demo client is found in the Fedora 2.0 distribution at: {FEDORA_HOME}/client/demo/soapclient. Programmers should examine the file DemoSOAPClient.java. To compile the demo, run the script named "compile-demo-soapclient.bat" (or .sh on unix). To run the demo, run the script named "run-demo-soapclient.bat" (or .sh on unix).

  8. Third party Library Upgrades

    All of the major third party libraries used in the Fedora codebase have been updated to reflect the most current stable versions available. This includes updating the underlying tomcat server to tomcat version 5. For Additional details regarding new library versions, refer to the license.html file in the distribution which has a complete list of the new libraries.

  9. Performance Tuning

    Significant enhancements have been made to the Fedora server code to improve ingest performance. These enhancements included both tuning of the underlying relational database as well as tuning how ingests are handled internally in the Fedora server.

  10. API-A and API-A-LITE Changes

    A general cleanup of the methods found in API-A and API-A-LITE has been done to provide better consistency between the SOAP-based and REST-based interfaces for API-A. The corresponding WSDL files have also been updated to reflect the changes.

    Previously in Fedora 1.2.1:

      API-A
  
    describeRepository
    findObjects
    getBehaviorDefinitions
    getBehaviorMethods
    getBehaviorMethodsXML
    getDissemination
    getObjectMethods
    getObjectProfile
    resumeFindObjects
 
  API-A-LITE

    describeRepository
    findObjects
    getDissemination
    getObjectProfile
    resumeFindObjects

    In Fedora 2.0:

      API-A

  describeRepository
  findObjects
  getDatastreamDissemination
  getDissemination
  getObjectHistory
  getObjectProfile
  listMethods
  listDatastreams
  resumeFindObjects

  API-A-LITE

  describeRepository
  findObjects
  getDatastreamDissemination
  getDissemination
  getObjectHistory
  getObjectProfile
  listDatastreams
  listMethods
  resumeFindObjects

    API-A Methods removed:

    • getBehaviorDefinitions - the functionality of this method is replaced with the revised getObjectProfile method which now provides complete object info including behavior definitions and methods.
    • getBehaviorMethods - the functionality of this method is replaced with the revised getObjectProfile method which now provides complete object info including behavior definitions and methods.
    • getBehaviorMethodsXML - the functionality of this method is replaced with the revised getObjectProfile method which now provides complete object info including behavior definitions and methods.
    • getObjectMethods - the functionality of this method is replaced with the revised getObjectProfile method which now provides complete object info including behavior definitions and methods.

    API-A Methods added or modified:

    • getDatastreamDissemination - this new method replaces the functionality provided in earlier releases using the getItem method of the Default Disseminator to return the contents of a datastream. Use of the default disseminator getItem method is now deprecated.
    • getObjectHistory - this new method provides a change history for a digital object listing each modification date when an object component has been modified. This is an interim solution to provide some basic information through API-A and API-A-LITE about the change history of an object. Eventually we would like to provide more detailed information from the Fedora audit records so one could obtain a detailed history of what changes were made to specific components of the object.
    • getObjectProfile - this method has been modified to return the complete object profile for a digital object. The object profile now includes all component information about an object.
    • listDatastreams - this new method replaces the functionality provided in earlier releases using the getItemIndex method of the Default Disseminator to list the datastreams of an object.
    • listMethods - this new method replaces the functionality provided in earlier releases using the getMethodIndex method of the Default Disseminator to list the methods of an object.
  11. API-M changes
    • logMessage parameter now exists on all object-modifying methods.
    • Datastream altIDs attribute. Any number of alternate identifiers can be specified for datastreams on creation or modification. This information is currently maintained, but not acted upon.
    • Datastream versionable attribute. A new boolean parameter that indicates whether Fedora should make changes in-place or retain prior versions on datastream modification. This information is currently maintained, but not acted upon.
    • New ingest method added. With the introduction of FOXML and support for multiple ingest formats, an additional parameter was required on ingest to specify the format of the file to be ingested. Valid values for format are "foxml1.0", "metslikefedora1", or "default". The format of "foxml1.0" specifies ingest files is encoded using FOXMLv1.0. The format of "metslikefedora1" specifies the encoding is Fedora's extension of METS. The format of "default" indicates to use the current repository default format. Refer to the API-M WSDL for additional information about the change. The former ingestObject method is now deprecated.
    • New export method added. With the introduction of FOXML and support for multiple export formats, an additional parameter was required on export to specify the format of the file to be exported and to specify the context of the export. Valid values for format are "foxml1.0", "metslikefedora1", or "default". Valid values for format are described above. Valid values for context are "public", "migrate", and "archive". A value of "public" produces an export file that is appropriate for use outside of the repository (all datastream content can be obtained via public callback URLs). A value of "migrate" produces an export file that is appropriate for migrating an object from one Fedora repository to another. A value of "archive" produces an export file that is a standalone entity, where all datastream content is inlines (XML inline and binary base64-encoded inline). Note that the "archive" option is not implemented in Fedora 2.0. Refer to the API-M WSDL for additional information about the change. The former exportObject method is now deprecated.
    • modifyDatastream now supports changing the mimeType and formatURI, and altID.
    • modifyDatastream and modifyDisseminator methods now support a "force" parameter, which indicates that even if a data type contract would be broken, Fedora should accept the change anyway.
    • purgeObject and purgeDatastream now include force parameters. Currently when specified as true, these throw an exception. This feature will be implemented in a future release.
    • Consistent parameter ordering. Parameter ordering of all the API-M methods was adjusted to make parameter ordering more consistent. Generally, the first one or two parameters identify the thing being accessed/modified (object PID, datastream or disseminator ID). When "logMessage" is relevant, it occurs just after any data parameters. When "force" is relevant, it occurs last.
    • The treatment of nulls versus empty strings in arguments for the modifyDatastream and modifyDisseminator methods of API-M is now more consistent. Arguments specified as null indicate that the original attribute of the component is to remain unchanged. Arguments specified as the empty string indicate the original attribute is to be set to the empty string. Arguments specified as non-null or non-empty indicate the original attribute is to be replaced by the specified value.

    Summary of API-M changes:

    • addDatastream: added parameters altIDs and logMessage
    • addDisseminator: added parameter logMessage; removed
    • parameters bDefLabel and bMechLabel
    • describeUser: [no change]
    • export: added parameters format, context
    • exportObject: [no change]
    • getDatastream: [no change]
    • getDatastreamHistory: [no change]
    • getDatastreams: [no change]
    • getDisseminator: [no change]
    • getDisseminatorHistory: [no change]
    • getDisseminators: [no change]
    • getNextPID: [no change]
    • getObjectXML: [no change]
    • ingest: added parameter format
    • ingestObject: [no change]
    • modifyDatastreamByReference: added parameters altIDs, versionable, MIMEType, and formatURI, and force
    • modifyDatastreamByValue: added parameters altIDs, versionable, MIMEType, formatURI, and force
    • modifyDisseminator: added parameter force; removed parameters bDefLabel and bMechLabel
    • modifyObject: [no change]
    • purgeDatastream: added parameters logMessage and force
    • purgeDisseminator: added parameter logMessage
    • purgeObject: added parameter force
    • setDatastreamState: [no change]
    • setDisseminatorState: [no change]
  12. PID Definition Syntax Relaxed

    Prior to Fedora version 2.0, the syntax for persistent identifiers (PIDs) was defined as a subset of the URN syntax, but was not strictly enforced by the server software. In version 2.0, the PID definition syntax has been relaxed to enable a wider range of possible PID values and the server software now strictly enforces the syntax of incoming PIDs.

    The following describes the syntactic constraints for PIDs in normalized form. The only differences with non-normalized PIDs are that the colon delimiter may be encoded as "%3a" or "%3A", and hex-digits may use lowercase [a-f].

      PID:
    
  Length : maximum 64
    
  Syntax : namespace-id ":" object-id
  
  namespace-id:
    
  Syntax : ( [A-Z] / [a-z] / [0-9] / "-" / "." ) 1+
  
  object-id:
    
  Syntax : ( [A-Z] / [a-z] / [0-9] / "-" / "." / "~" / "_" / escaped-octet ) 1+
    
  
  escaped-octet:
  Syntax : "%" hex-digit hex-digit
  
  hex-digit:
  Syntax : [0-9] / [A-F]
  13. Including PID As Default Parameter in Disseminations

    When constructing complex user-defined disseminators that reference disseminations from other objects, it is often handy to be able to pass the PID of the data object on to the nested dissemination requests. Prior to Fedora 2.0, the only way to accomplish this was to add a formal User Parameter to the disseminator method and use it to pass the value of the PID. This syntax is awkward since the PID of the data object is already contained in the dissemination request. For example:

    http://localhost:8080/fedora/get/demo:999/demo:991/getMethod?userparm1=value&userparm2=value&pid=demo:999

    In the future, we plan to make it possible for the PID of the data object to be automatically passed as a parameter so it is always available downstream as a valid parameter on any dissemination request. As an interim solution to address this issue, Fedora 2.0 allows the PID of the data object to be passed as a default behavior mechanism parameter by using a special syntax of the DefaultInputParm element in the behavior mechanism object. For example:

    <fmm:DefaultInputParm defaultValue="$PID" label="The PID of the data object" parmName="PID" passBy="VALUE" required="true"/>

    Specifying a DefaultInputParm with a default value of "$PID" will result in the PID of the data object being passed on to the external service defined in the behavior mechanism object. In the WSDL section of the behavior mechanism object, reference the special DefaultInputParm as you would other parameters. For example:

      <wsdl:operation name="getService">
    <http:operation location="runScript.pl?pid=(PID)"/>
       .
       .
       .

    At run time, this will be translated into

      runScript.pl?pid=pid-of-the-data-object

    A similar syntax is available to pass the object URI of the data object. If you want to pass the data object's URI, then specify a DefaultInputParm with a default value of "$objuri" and this will be translated at run time into

    runScript.pl?pid=info:fedora:/pid-of-the-data-object

    where PID is the persistent identifier of the data object.

    The demo data object object named demo_SmileyStuff.xml and the associated behavior mechanism object named demo_DualResImageCollection.xml provide an example of the new syntax.

  14. Syntax Of Timestamp Values Extended

    Prior to Fedora 2.0, the syntax of timestamp values was to the nearest second. This range has been extended in Fedora 2.0 to include milliseconds to capture versioning changes that occur in less than one second. The new timestamp syntax is:

      YYYY-MM-DDTHH:MM:SS:SSSZ
  
  e.g., 2005-01-17T12:13:54:185Z
  15. Redesign of Fedora Web Site and Documentation

    The Fedora web site at http://www.fedora.info has been totally redesigned to provide enhanced readability, better navigation, and a more consistent "look and feel". Areas like the "Tools" section have been added to provide a place where Fedora users can share code and techniques they have developed using Fedora. The tools section is still under construction but will be available for accepting posts in the near future.

  16. Fedorial Tutorials

    A new series of tutorials is being developed that provides step-by-step instuctions for accomplishing common tasks with Fedora. The tutorials target a broad audience from system developers and repository managers to end users. Currently, there are three planned tutorials in the series:

    1. Introduction: Basic Concepts in Fedora
    2. Getting Started: Creating Fedora Objects and Using Disseminators
    3. Information Modeling: Designing Fedora Content Models
    As of Fedora 2.0, only the first two tutorials are available. The third tutorial is in progress and will be available in the spring of 2005. The tutorials can be found in the release in the tutorial directory under userdocs. The tutorials are also available on the web site at http://www.fedora.info/documentation/tutorials.shtml.

C. Bug Fixes

  1. Incorrect GMT Datetimes - Creation dates for objects do not appear to have the correct GMT timestamps. (Bugzilla #52)

    When objects were created, the GMT timestamps stored did not reflect the correct time of object creation. This was bug in fedora.server.utilitites.DateUtility. The time zone offset was incorrectly being applied to the creation datetime. Objects now reflect the correct GMT creation datetimestamp.

  2. Managed datastreams inadvertently deleted - Under certain situations managed content datastreams could be deleted when purging similarly PIDed objects. (Bugzilla #53)

    This was a bug in fedora.server.storage.replication.DefaultDOReplicator. Managed datastreams are stored internally using an identifier that consists of objPID + dsID + dsVerID. An errant regular expression was being greedy in matching which could result in similarly PIDed objects being deleted. The regular expression is fixed to prevent this from occuring.

  3. Errant space in SchemaLocation used by getnextPID - There is an erroneous space preceding the SchemaLocation in output for getNextPID. (Bugzilla #54)

    The SchemaLocation in the xml header output when running the getNextPID method of API-M and API-M-LITE contains an errant blank. This bug will only cause problems if trying to validate the xml output from the server. The errant space has been removed.

  4. Dublin Core description element missing from drop down menu in admin GUI client. (Bugzilla #55)

    The drop down menu for Dublin Core elements in the admin GUI does not contain the description element. The description element has been added to the drop down list.

  5. Unable to disseminate timestamped disseminations - Under certain situations, attempting timestamped disseminations would throw NullPointerException. (Bugzilla #56)

    This bug only occurs when attempting timestamped disseminations using user-defined disseminators (i.e., disseminators other than the default disseminator).

    This was a bug in fedora.server.storage.SimpleDOReader. When datastream state checking was added in Fedora 1.2, the section of code dealing with populating the DisseminationBindingInfo class did not include the new variable for datastream state. As a result this variable in the class is never initialized which leads to the eventual null pointer exception when attempting timestamped disseminations with user-defined disseminations.

  6. Serializer fails on export of bMech objects. Exporting bMech objects throws NullPointerException. (Bugzilla #57)

    The problem was that the method that checked WSDL and SERVICE-PROFILE datastreams for instances of the local pattern "http://local.fedora.server" did not have this pattern compiled. This caused a null pointer exception when the xml datastreams were searched for this pattern. Bmech objects now export correctly.

  7. Object Label updates not reflected in ObjectProfile - Changing an object's label in the admin GUI is not reflected when disseminating objectProfile. (Bugzilla #58)

    This was a bug in the replicator where the object label field was not being updated when the object label was changed in the object xml. Object label changes are now properly displayed when disseminating the objectProfile.

  8. Timestamp information not displayed in all views of Default Disseminator (Bugzilla #59)

    Timestamp information was not being properly handled by the XSL stylesheet that is used to style the output from viewItemIndex. This is now corrected and timestamp info is properly displayed in all views of the default disseminator (getObjectProfile, viewItemIndex, and viewMethodIndex).

  9. Date issues when Server and Client in different Timezones - Creation dates for objects can be incorrect when server and client are in different time zones. (Bugzilla #60)

    When a Fedora server is in one time zone and a Fedora client is in a different time zone, it was discovered that incorrect timestamps were displayed by the client when retrieving objects from the server.

    If you open the object editor on an existing object and go to the Datastreams pane, you will see that the created dates on the version timeline do not match what's in the datastream at the server.

    Even though the admin GUI client is getting the creation data from the server (via the java Calendar and Date classes), it appears that the client evaluation of the values considers the system clock current settings which are adjusted based on time zone. This is probably embedded in the underlying implementation of the java Calendar class.

    The fix was to redesign the handling of timestamp information by both the Fedora client and server. Fedora now standardizes on all dates PRODUCED by Fedora as STRINGS in ISO8601 format: mm-dd-yyThh:mm:ssZ. This allows Fedora to accept strings in mm-dd-yy or mm-dd-yyThh:mm:ss or mm-dd-yyThh:mm:ssZ format, to keep backward compatiability.

  10. Problem when versions get assigned the same timestamp (Bugzilla #64)

    In certain situations when modifications to an object component occur in rapid succession it was observed that a timestamp granularity of seconds was insufficient and you could have two versions with the same timestamp values to the nearest second. The solution was to extend the value of timestamps to include milliseconds to prevent any potential conflict.

  11. Inconsistent treatment of nulls versus empty string in arguments of modifyDatastream and modifyDisseminator methods of API-M (Bugzilla #68)

    The treatment of nulls versus empty strings in arguments for the modifyDatastream and modifyDisseminator methods of API-M is now more consistent. Arguments specified as null indicate that the original attribute of the component is to remain unchanged. Arguments specified as the empty string indicate the original attribute is to be set to the empty string. Arguments specified as non-null or non-empty indicate the original attribute is to be replaced by the specified value.

  12. Error when editing disseminator label when using McKoi (Bugzilla #76)

    When attempting to edit a disseminator's label when using McKoi as the underlying SQL database, the server would throw SQLException. This error was due to a typographical error in the database column name being used. Oracle and MySQL are case insensitive regarding column names, but McKoi is case sensitive. This allowed dissemination labels to work under MySQL or Oracle implementations, but fail with McKoi implementations. The case of the column name is now corrected and works for all three database implementations.

  13. Oracle error on ingest: cannot insert NULL into DSBIND.DS (Bugzilla #77)

    In certain situations when using Oracle as the underlying SQL database, ingest would fail with SQLException. The cause was attempting to insert NULL into the column dsBindKeySeq when no value was specified in the object for this optional value. Oracle does not allow NULL for this column type so the code has been modified to insert zero("0") when dsBindKeySeq is not specified in the object.

  14. Editing object label containing apostrophe throws SQLException (Bugzilla #82)

    When attempting to edit an object's label containing an apostrophe, the server responds with SQLException. The apostrophe character is a special character that must be properly escaped in SQL INSERT statements. Object labels are now checked for special characters and properly escaped before issuing the SQL INSERT command.

  15. Oracle char(1) column types return with space padding (bugzilla #85)

    It was observed that when using Oracle as the underlying SQL database, column types of char(1) would return with space padding which could result in errors downstream. This was not the case with McKoi or MySQL. The fix was to change the db schema column type to use varchar(1) instead of char(1) everywhere in the db schema.

  16. Object label containing ampersand character throws Exception when attempting to disseminate getObjectProfile method (bugzilla #86)

    It was observed that creating an object with a label containing the ampersand character would later cause the dissemination of the getObjectProfile method to fail. This resulted from special characters, like the ampersand character, not being properly escaped when written to dynamically generated xml served by Fedora. XML special characters are now properly escaped before being written to the outgoing xml stream so ampersands and other special xml characters can safely be included in object labels.

 

3. Release 1.2.1 (Bug Fix Release) - April 19, 2004

Fedora 1.2.1 provides fixes to several bugs that were reported by users of Fedora 1.2. The new release is backward compatible with Fedora 1.2 and contains no changes to the underlying database schema.

IMPORTANT!! - Migrating From Fedora Releases Prior to Fedora 1.2

Users wishing to migrate from releases prior to Fedora 1.2 need to follow the steps outlined in Release 1.2 - December 15, 2003 and migrate to version 1.2 BEFORE attempting migration to Fedora 1.2.1.

Migrating From Fedora 1.2 to Fedora 1.2.1

If you are upgrading from Fedora 1.2:

BUG FIXES:

1. Errors in installation document regarding source installations. (Bugzilla # 3)

The installation document has been updated to say that the JAVA_HOME environment variable must be set prior to installing the source distribution since ant requires this environment variable to perform the build. A typographical error regarding changing the permissions on the ant executables has been corrected to read: "This can be done with the command: chmod 755 $FEDORA_DEV/res/ant/bin/*."

2. Admin GUI client - Purge Failure error message text does not wrap. (Bugzilla # 5)

When an error is encountered while attempting to purge an object, the error message displayed in the message pane does not properly wrap and scrolls off the edge of the window. This has been modified to properly wrap the message text so the complete message is visible regardless of length.

3. Admin GUI client - Purging object does not close object paneI in ObjectEditor. (Bugzilla # 6)

When viewing an object using the ObjectEditor and then clicking the purge button, the object is removed from the repository, the object pane remains on the screen. The admin GUI client has been modified so that the object pane is now closed when the object is purged.

4. Admin GUI client - New Dialog Enhancement Request . (Bugzilla # 7)

In the New object creation dialog, one enters label text and content model text and clicks "create" to ingest the object. Pressing the ENTER key instead of clicking the "create" button has no affect. This has been changed so pressing the ENTER key now has the same result as clicking the "create" button on NewObjectDialog pane.

5. OAI day-granular "until" parm taken as midnight beginning that date. (Bugzilla # 9)

For OAI requests, the code interprets both "from" and "until" parms (which both have day-granularity) as meaning midnight beginning that date. This is correct for the "from" parm only. The day-granularity "until" parm should be interpreted as "23:59:59pm ending that date". That way, an OAI request with equal day-granular "from" and "until" parms will return records for that date. Changes have been made regarding the way that the OAI "until" date parm is handled when using "Day" granularity. If Day granularity is being used, the date value for the Until date parm is modified to be Day + 23:59:59. This means that when using Day granualrity, specifying the same Day for both "from" and "until" date parms will result in matching all dates between 00:00:00 and 23:59:59 on that day.

6. Admin GUI client - Export dialogs are incomplete under MAC OSX implementation. (Bugzilla # 10)

This problem shows up only on MACs in exports from an opened object. In contrast, the dialogs from both File and Export submenus (export one object, export by type) do work correctly, as with Windows. These correct cases are where the filename is selected programmatically (i.e., pid.xml). Exports from an opened object (either to export the object as xml or to export a datastream of the object) give dialog boxes which lack filename fields so the user has no opportunity to select the filename. Navigating with the dialogue to a directory with an existing file needs to match the needed mimetype, and selecting that file will result in overwriting that file's contents with either the object xml or the datastream contents, but without changing the filename. The JFileChooser dialog was not allowing entry of filenames in cases where that should be possible. The workaround was to, for these specific cases, default to the platform's native file dialog by using java.awt.FileDialog. Note that using the platform's native dialog is not a solution for all file dialog-related actions. JFileChooser still needs to be used in some cases because it has an option to only allow the choosing of directories, whereas FileDialog is limited in this regard.

7. Admin GUI client - Log file generated when ingesting objects by type is incorrectly persistent. (Bugzilla # 11)

When performing multiple ingest objects by type operations in the same admin client session, the log file displayed for subsequent ingest objects all show the log file from the initial ingest object operation. Closing the admin client between ingest object operations produces the expected results. This behavior is a result of the code checking to see if the log PrintStream is null to decide if a new log file is to be generated. If null, a new log is generated, otherwise it assumes there is an existing log to be displayed. Although the closeLog method in fedora.client.ingest.Ingest closes the log PrintStream(s_log), it does not get immediatley garbage collected so can have a non-null value for some time period after being closed. The code has been modified to explicitly set the PrintStream to null after being closed to resolve this issue.

8. Admin GUI client - After deleting disseminator using ObjectEditor, the bdef drop down list is not refreshed. (Bugzilla # 12)

If you open an existing object, delete a disseminator, then try to re-build the disseminator in the same session, the drop down list of behavior definitions does not include the bdef that was associated with the disseminator that was just deleted. If you exit the object editor after deleting the disseminator, then re-open the object in the editor, then the bdef becomes available again in the drop down. The code has been updated to refresh NewDisseminatorPane after a purge so the bdef used by the purged disseminator is visible once again in the drop down list of bdefs.

9. Fedora search not accepting underscores. (Bugzilla # 13)

The use of the underscore (_) character in Fedora searches generates an error that varies depending on the sql database being used. There was some obsolete code involving the syntax of the WHERE clause for Search queries where a special string " { escape '/'} " was appended to the end of any WHERE clause search string containing special characters that required escaping according to the SQL92 syntax rules. This code is no longer required and has been removed so underscores in searches no longer produce an error.

10. WSDL documentation - Inline wsdl docs for api-m.ingestObject mention incorrect state. (Bugzilla # 14)

The wsdl docs inside the Fedora-API-M.wsdl file say that upon ingest, an object's original state is "N", which is wrong. It should be "A". The comments in API-M.wsdl for ingestObject and createObject have been updated to reflect that the initial state of object is set to "A".

11. API-M addDisseminator method - addDisseminator method fails to update sql db when more than one pre-existing disseminator. (Bugzilla # 16)

When creating an object using the admin GUI client that has one or more existing disseminators, adding a new disseminator adds the disseminator to the object xml, but fails to properly replicate the disseminator in the sql database. This was a bug in DefaultDOReplicator that caused improper updating of the sql database in the case where a new disseminator was added to a data object, via the addDisseminator method of API-M, where the object had one or more pre-existing disseminator(s). The result was a mismatch between the xml representation of the object and the sql database representation. This problem has been corrected.

12.Admin GUI client - List of MIME types for XMLMetadata displays types other than text/xml when adding multiple datastreams. (Bugzilla # 17)

In the Admin GUI client, the MIME type drop down menu should only list text/xml when the XMLMetadata radio button is checked. This is true when the "new Datastream" button is clicked for the first datastream. However, if the datastream is saved, and another new datastream added, the MIME type list shows MIME types other than text/xml when the XMLMetadata radio button is checked. The code has been modified so that only MIME tpyes of text/xml are displayed for datastreams of type XMLMetadata.

14. Demo Objects - Label for DC datastream of demo object obj-ead-finding-aid.xml incorrect. (Bugzilla # 20)

The demo object (demo:17) for the Rita Mae Brown finding aid has an incorrect label when you do a viewItemIndex. The DC record's label is shown as "DC Record for Pavillion III Architectural image object" which is wrong. However, when you click on that link, the underlying xml looks correct. The label for demo object demo:17 has been corrected.

15. Search - resumeFindObjects skips next result in sequence. (Bugzilla # 21)

When displaying search results that spanned more than one screen, the search interface will skip over the first entry on the next screen. For example, if the maximum number of search hits to display is 10 and there are 30 hits, the search will display hits 1-10, then 12-21, and then 23-30 omitting hits 11 and 22. This was a bug in fedora.server.search.FiledSearchResultSQLImpl where the cursor in the ResultSet was being positioned incorrectly when there are more results to display than the specified maxResults value. The cursor is now correctly positioned and all hits are displayed.

16. Admin GUI client - Adding/removing datastream bindings in Admin GUI client result in broken disseminations. (Bugzilla # 24)

Removing and then re-adding a datastream binding in admin gui for a disseminator breaks the dissemination for that disseminator. XML representation of the object appears correct, but disseminations handled through sql database are broken. This was a bug in fedora.server.storage.DefaultDOReplicator that had to do with the improper updating of dsBind table when the binding map of an existing object was altered. The xml was correct after the update, but replication was incomplete on the sql database. The end result before the fix was an incomplete update of the dsBind table which had the effect of breaking disseminations for that object because of missing bindings in dsBind table. This has been corrected.

17. Admin GUI client -The state of datastreams is reset to Active in sql db when binding datastreams to a disseminator. (Bugzilla # 25)

If a datastream has a state other than Active and it is bound to a disseminator or if already bound and the binding is removed/re-added, the state of the associated datastream is reset in the sql db to Active. This was a bug in fedora.server.storage.DefaultDOReplicator that had to do with the updating of dsBind table when creating or modifying datastream bindings. The datastream state is correct in the xml, but the replication is incorrect in the sql database. Before the fix, the state of the updated bindings was always being set to "A" so if the state of a datastream was other than "A" and a binding that used that datastream was created/modified, the state in the sql database was reset to "A". The state of the datastream in the xml is now used to set the state when creating or updating a binding map. This guarantees that the state in dsBind table will always be the sameas that of datastream in xml.

18. Batch Utiltity - Sample mets-template.xml used in batch utility demo contains metadata wrapper errors. (Bugzilla # 25)

Some of the METS metadata wrapper elements in the mets-template.xml example file used with the batch utility are incorrect. e.g., administrative source metadata is in a techMD wrapper element. The typographical errors have been corrected in the sample template file.

19. Oracle open_cursor limit exceeded (Bugzilla # 29)

Users implementing Fedora with Oracle reported that the database would hang when attempting deletes of large numbers of objects. The database hang was the result of exceeding the open_cursor limit set by the database. This problem has only been reported with Oracle. There are apparently some subtle differences between the various JDBC drivers and how they handle cleanup when a ResultSet or Statement is closed. The JDBC code has been modified to better handle these variations and address the open_cursor problem for Oracle users.

20. FedoraMets ID lengths - ID lengths silently ignored. (Bugzilla # 30)

Fedora stores the definitive copy of objects as xml, but for performance also replicates a copy of objects in an sql database. With xml, there are no limitations on the length of attribute values or content. However, the sql database does impose field lengths which restrict the length of identifiers and labels. Users reported the truncation of some identifers in the sql database as a result of having identifier strings in the xml that exceeded the database schema field lengths. The xml validation code has been modified to verify the length of Fedora identifiers. Exceptions are now thrown at ingest time if any identifiers in the xml exceed the field lengths specified for those fields in the database schema. Currently, the validation is only applied to identifiers. Descriptive fields like labels, will be be truncated if they exceed the database schema field lengths. We will consider extending the validation checking to labels in the next release.

 

 

4. Release 1.2 - December 15, 2003

This new release contains a number of significant new features, including content versioning, the complete implementation of the Fedora Management interface, and major additions to the Fedora Administrator client to enable object creation and editing. With the advent of content versioning, the Fedora Access interfaces now support date-time stamped requests, so that a client can "go back in time" and see a digital object as it looked in the past. Additionally, this release provides a migration utility for mass export and mass ingest of objects from either directories or other repositories. The migration utility enables the moving of objects from older versions of Fedora repositories into the lastest version. It also is of general utility for copying or moving objects among repositories, for unloading repositories, and for bulk ingest of objects.

IMPORTANT!!! Upgrading From a Previous Release of Fedora

Although Fedora 1.2 is backward compatible with previous versions of Fedora at the interface definition level, there are changes to the underlying database schema that require digital objects to be migrated from an old Fedora installation to the new Fedora installation. If you want to keep digital objects from an old repository and migreate them to the new Fedora 1.2 repository, you must run the migration utility provided with Fedora 1.2. DO NOT INSTALL FEDORA 1.2 ON TOP OF THE PREVIOUS RELEASE. To upgrade from a previous release and migrate digital objects into the new repository follow these steps:

STEP 1 - INSTALLATION:

Follow the steps outlined in the Fedora Installation Guide. If you plan to migrate digital objects from a repository that is a previous version of Fedora, be sure to configure the new Fedora 1.2 installation to run in parallel to the existing installation. After the new Fedora 1.2 is installed, you will run a migration utility that will unload all objects from your old repository and ingest them into your new repository. If you are installing Fedora 1.2 on the same machine as a previous version, make sure to set the following configurations for the Fedora 1.2 installation:

1. install Fedora 1.2 into a different directory than the previous installation (set $FEDORA_HOME environment variable accordingly)

2. initialize a new database for Fedora 1.2 using the database configuration scripts which are set up with new defaults. If using MySQL, run mysql-config.bat (.sh for unix) with the new default database name of "fedora12" (or your own new database name that differs from the name used in your pre-existing Fedora installation. For McKoi, run the mkoi-init.bat (.sh for unix) which has a new default port number for Fedora 1.2.

3. set a different port for Fedora 1.2 to run on (fedoraServerPort in the Fedora config file fedora.fcfg)

4. set a different shutdown port for Fedora 1.2 (fedoraShutdownPort)

5. set a different redirect port for Fedora 1.2 (fedoraRedirectPort)

6. set a different storage path for digital objects (object_store_base)

7. set a different storage path for datastreams (datastream_store_base)

8. set a different storage path for temporary files (temp_store_base)

9. set the Fedora 1.2 repository to allow the PIDs from the old repository to be accepted (add namspaces to retainPIDs in fedora.fcfg)

10. if the OLD Fedora installation is running on a different machine, modify the fedora configuration file of the OLD Fedora installation to requests to be made by the new Fedora 1.2 repository. (Add IP address of Fedora 1.2 repository to allowHosts in fedora.fcfg of OLD repository.)

STEP 2 - MIGRATION:

Follow the instructions in the Fedora Migration Utility Guide. to migrate digital objects from your previous Fedora installation to the new Fedora 1.2 repository. The new repository migration utility is provided with Fedora 1.2 to easily perform a mass export and mass ingest of objects from the old repository to the new repository. The repository migration utility can copy or move objects among Fedora 1.1, Fedora 1.1.1, and Fedora 1.2 repositories. If you are upgrading from Fedora 1.0, please first upgrade to Fedora 1.1 before installing Fedora 1.2 since the migration utility does not work with Fedora 1.0.

NEW FEATURES:

1. Fedora Management Interface (fully implemented).

In Fedora 1.2 all operations of the Fedora management interface (API-M) are implemented. Operations for creating, modifying, inactivating, and deleting components of digital objects (i.e., Datastreams and Disseminators) are now available in the management web service via SOAP bindings.

2. Content Versioning.

The Fedora content versioning system is enabled with the release of Fedora 1.2. Now, any modifications made to a Datastream or Disseminator through the Fedora management interface (API-M) will automatically result in a new version of that Datastream or Disseminator being created by Fedora. The Fedora repository maintains all versions of all Datastreams and Disseminators, thereby creating a history of how objects change over time. Additionally, Fedora maintains an audit trail record of the nature of the object change events (e.g., who, what, when, why).

3. Fedora Administrator - Object Editor and Version Viewer.

The new content versioning features can be easily accessed via new menu items in the Fedora Administrator client. The new object editor features (i.e., File/New, File/Open) provide the ability to create and maintain digital objects through a graphical user interface. Using these menu items, the user can create a new digital object, as well as add/modify/delete Datastreams and Disseminators, and view the different versions of these components. For details see the Fedora Administrator Client documentation and the Fedora Content Versioning documentation.

4. Date-Time-stamped Access Requests.

Building on the capabilities of the new content versioning features, the Fedora repository will now support time-stamped disseminations and API-A requests. This enables a client accessing Fedora to be able to request a view of a digital object as it looked at a particular point in time. The syntax for using date-time in access requests is doc