/tmp Folder 100% Full in Linux – Complete Troubleshooting Guide for Linux & Oracle DBAs
/tmp Folder 100% Full in Linux – Complete Troubleshooting Guide for Linux & Oracle DBAs
A 100% full /tmp directory is one of the most common Linux administration problems encountered by System Administrators, Oracle Database Administrators, Oracle E-Business Suite Application DBAs, and DevOps Engineers.
Although it may initially appear to be a simple disk space issue, a full /tmp filesystem can prevent applications from creating temporary files, cause software installations to fail, interrupt Oracle patching activities, stop Oracle E-Business Suite services, and even prevent users from logging into the operating system.
Oracle Database installations, Oracle Grid Infrastructure, Oracle E-Business Suite cloning, OPatch, AutoConfig, RMAN operations, Java applications, package managers, and numerous Linux utilities all rely on the /tmp directory for temporary storage.
When this filesystem becomes full, administrators often delete files immediately to reclaim space. However, deleting the wrong files while applications are still using them can create additional problems and may even interrupt production services.
This guide explains how the Linux /tmp directory works, why it becomes full, how to diagnose the real cause safely, and the production-tested procedures used by experienced Linux and Oracle DBAs to resolve the issue without affecting running applications.
Before deleting any files, determine whether the problem is caused by large temporary files, excessive inode usage, a tmpfs filesystem reaching its memory limit, or files that remain open after deletion. Review filesystem usage with df -h, check inode consumption using df -i, identify large files with du, and verify open deleted files using lsof. Only remove temporary files that are no longer required and are not being used by active processes.
Typical Error Messages
A full /tmp directory may generate different errors depending on the application.
No space left on device Disk quota exceeded Cannot create temporary file Unable to create temp file Write failed mktemp: failed to create file java.io.IOException: No space left on device OPatch failed AutoConfig failed Oracle Universal Installer failed RMAN backup failed
What is the /tmp Directory?
The /tmp directory is a standard Linux filesystem location used to store temporary files created by the operating system and applications.
Files stored in /tmp are generally intended to exist only for a short period. Many Linux distributions automatically remove old temporary files during reboot or through scheduled cleanup services.
Common users of the /tmp directory include:- Linux operating system utilities
- Oracle Database installation programs
- Oracle Grid Infrastructure
- Oracle E-Business Suite AutoConfig
- Oracle OPatch
- RMAN operations
- Java Virtual Machine (JVM)
- Package managers such as yum, dnf, and apt
- Shell scripts
- Web servers
- Application servers
How Linux Uses /tmp
During normal operation, applications create temporary working files inside /tmp. These files may include extracted installation packages, cache files, session data, socket files, logs, temporary backups, lock files, and intermediate processing results.
After processing completes, well-designed applications remove these temporary files automatically. However, abnormal application termination, unexpected server shutdowns, software bugs, or insufficient housekeeping can leave thousands of unnecessary files behind.
Common Causes of a Full /tmp Directory
- Large temporary files created by applications.
- Oracle installation files left after software installation.
- OPatch temporary extraction directories.
- Oracle E-Business Suite cloning activities.
- AutoConfig temporary files.
- RMAN temporary processing files.
- Java temporary files.
- Core dump files.
- Large log files.
- Application cache files.
- Thousands of very small files consuming inodes.
- Files deleted while still opened by running processes.
- Memory-based
tmpfsfilesystem reaching its configured limit. - Applications failing to remove temporary files after abnormal termination.
Business Impact
A completely full /tmp directory can affect both Linux and enterprise applications.
Common business impacts include:- Oracle software installation failures.
- Oracle patching interruptions.
- Oracle Database upgrade failures.
- Oracle E-Business Suite AutoConfig failures.
- Oracle E-Business Suite cloning failures.
- RMAN backup interruptions.
- Application server startup failures.
- Java application crashes.
- Software package installation failures.
- User login failures.
- Production outages.
- Unexpected application downtime.
Why Simply Deleting Files May Not Solve the Problem
Many administrators immediately remove files from /tmp after seeing that the filesystem is full. While this may temporarily recover disk space, it does not always resolve the underlying issue.
For example, if a running process has already opened a temporary file, deleting the file removes only the directory entry. The operating system continues to reserve the associated disk space until the process exits or releases the file descriptor.
Similarly, if the filesystem has exhausted its available inodes, deleting only a few large files may have little effect because the limitation is the number of files rather than their size.
For these reasons, experienced Linux administrators always diagnose the exact cause before performing cleanup.
Where Should You Start?
The first objective is to determine why the filesystem reports 100% usage.
A structured investigation should answer questions such as:- Is the filesystem truly full?
- Is the problem caused by disk space or inode exhaustion?
- Is
/tmpmounted as tmpfs? - Which files consume the most space?
- Are deleted files still held open by running processes?
- Is Oracle software currently using temporary files?
- Has a recent installation, patch, backup, or cloning activity filled the directory?
Once these questions are answered, the appropriate cleanup strategy becomes much safer and significantly more effective.
Never delete the entire contents of /tmp on a production server without first verifying which applications are using the files. Oracle Database, Oracle E-Business Suite, Java processes, backup utilities, and operating system services may rely on temporary files during execution. Always diagnose the root cause first, then remove only files that are confirmed to be safe.
Step-by-Step Linux & Oracle DBA Troubleshooting
When the /tmp directory reaches 100% utilization, the objective is to determine why it is full before removing any files. Production environments often contain active Oracle processes, application servers, Java applications, and operating system services that depend on temporary files.
Step 1 – Verify Filesystem Usage
First, confirm whether the filesystem containing /tmp is actually full.
df -h /tmpExample output:
Filesystem Size Used Avail Use% /dev/sda3 20G 20G 0 100%If the filesystem shows 100% utilization, continue with further investigation.
Step 2 – Verify Inode Usage
Sometimes disk space is available, but all available inodes have been consumed by millions of small files.
Check inode usage:df -i /tmpIf inode usage is near 100%, identify directories containing excessive numbers of files.
Step 3 – Determine Whether /tmp Uses tmpfs
Many modern Linux distributions mount /tmp as a memory-backed tmpfs filesystem.
Verify:mount | grep /tmpor
findmnt /tmpIf
tmpfs is used, the available space depends on system memory rather than physical disk capacity.
Step 4 – Identify Large Files
Locate the largest files occupying space inside /tmp.
du -sh /tmp/*or
du -ah /tmp | sort -rh | head -20Focus on unexpectedly large files and directories.
Step 5 – Locate Recently Modified Files
Files created recently may indicate the application responsible for consuming disk space.
find /tmp -type f -mtime -1This lists files modified during the last 24 hours.
Step 6 – Detect Open Deleted Files
Deleting a file does not immediately release disk space if a running process still has the file open.
Check for open deleted files:lsof +L1or
lsof | grep deletedRestart or stop the associated process only after confirming it is safe to do so.
Step 7 – Verify Running Oracle Processes
Before deleting Oracle-related temporary files, verify whether Oracle software is actively using them.
Examples:ps -ef | grep pmon ps -ef | grep ora_ ps -ef | grep javaAvoid deleting temporary files while Oracle installations, upgrades, cloning, RMAN backups, or OPatch sessions are running.
Step 8 – Remove Unused Temporary Files Safely
After confirming files are no longer required, remove only unnecessary temporary files.
Example:find /tmp -type f -mtime +7 -deleteAlways verify the command before execution on production systems.
Step 9 – Review Oracle Installation Activities
Oracle software frequently creates temporary extraction directories inside /tmp.
Review whether the server recently performed:- Oracle Database installation
- Grid Infrastructure installation
- Oracle PSU or RU patching
- OPatch updates
- Oracle EBS cloning
- AutoConfig execution
- Rapid Install
Step 10 – Verify Filesystem Permissions
The /tmp directory should normally have world-writable permissions with the sticky bit enabled.
Verify:ls -ld /tmpExpected output:
drwxrwxrwtIncorrect permissions can prevent applications from creating temporary files.
Step 11 – Monitor Disk Growth
If the filesystem continues filling after cleanup, identify the application continuously generating temporary files.
Useful commands:watch df -h watch du -sh /tmpContinuous growth usually indicates an application bug or runaway process.
Step 12 – Review System Logs
System logs often identify the application responsible for abnormal temporary file generation.
Review:journalctl dmesg tail -100 /var/log/messages
Real Production Case Study
An Oracle E-Business Suite R12.2 production clone repeatedly failed during AutoConfig with the error:
No space left on device
Although several gigabytes of files had already been deleted from /tmp, the filesystem still reported 100% utilization.
Running lsof +L1 revealed that multiple Java processes were still holding deleted temporary files open. After the clone process was stopped and the Java processes terminated gracefully, the operating system immediately reclaimed the disk space. AutoConfig was rerun successfully and the clone completed without further issues.
Linux Administrator Checklist
| Verification | Status |
|---|---|
| Filesystem Usage Checked (df -h) | ☐ |
| Inode Usage Checked (df -i) | ☐ |
| tmpfs Verified | ☐ |
| Large Files Identified | ☐ |
| Open Deleted Files Checked | ☐ |
| Oracle Processes Verified | ☐ |
| Filesystem Permissions Verified | ☐ |
| Old Temporary Files Removed Safely | ☐ |
| System Logs Reviewed | ☐ |
| Issue Successfully Resolved | ☐ |
Linux Distribution Considerations
The behavior of the /tmp directory varies across Linux distributions. Some systems use a traditional disk-based filesystem, while others mount /tmp as a memory-backed tmpfs. Understanding this distinction is important because the troubleshooting approach differs.
| Linux Distribution | /tmp Implementation |
|---|---|
| Oracle Linux | Usually disk-based, but may use tmpfs depending on configuration. |
| Red Hat Enterprise Linux (RHEL) | Can be either disk-backed or tmpfs. |
| CentOS | Depends on system configuration and version. |
| Ubuntu | Often uses tmpfs on modern releases. |
| SUSE Linux Enterprise Server (SLES) | Supports both configurations. |
Preventive Best Practices
- Monitor filesystem usage regularly.
- Monitor inode utilization in addition to disk space.
- Configure automatic cleanup of old temporary files.
- Avoid storing permanent files in /tmp.
- Schedule periodic housekeeping tasks.
- Review Oracle installation and patching directories after maintenance.
- Monitor Java applications for excessive temporary file generation.
- Verify that applications remove temporary files after completion.
- Maintain sufficient free space for Oracle patching and upgrades.
- Include /tmp monitoring in enterprise monitoring tools.
Automatic Cleanup Options
Most modern Linux distributions include services that automatically remove stale temporary files.
Examples include:- systemd-tmpfiles
- tmpwatch
- tmpreaper
Common Administrator Mistakes
- Deleting the entire contents of /tmp without checking active processes.
- Ignoring inode usage.
- Assuming deleted files immediately free disk space.
- Removing Oracle installation files while patching is still in progress.
- Ignoring Java processes holding deleted files open.
- Overlooking
tmpfsmemory limits. - Failing to investigate repeated disk growth.
- Changing permissions on /tmp incorrectly.
- Using
rm -rf /tmp/*indiscriminately on production systems. - Not documenting cleanup activities.
Useful Linux Commands
Filesystem Usage
df -h
Inode Usage
df -i
Largest Directories
du -sh /tmp/*
Largest Files
du -ah /tmp | sort -rh | head
Open Deleted Files
lsof +L1
Mounted Filesystems
mount | grep /tmp
Filesystem Type
findmnt /tmp
Directory Permissions
ls -ld /tmp
Troubleshooting Flowchart
/tmp Reports 100% Full
│
▼
Run df -h
│
▼
Run df -i
│
▼
Disk Full or Inodes Full?
│
┌──────┴──────┐
│ │
Disk Inodes
│ │
Find Large Find Many
Files Small Files
│ │
▼
Check lsof +L1
│
▼
Active Process?
│
┌──────┴──────┐
│ │
Yes No
│ │
Stop Process Remove Files
│
▼
Verify Free Space
│
▼
Problem Resolved
Frequently Asked Questions (FAQ)
Is it safe to delete everything inside /tmp?
Not always. Active applications may still be using temporary files. Always verify running processes before deleting files from /tmp.
Why does disk space remain full after deleting files?
A running process may still have the deleted files open. Use lsof +L1 to identify such files and restart the associated process only when appropriate.
What is the difference between disk space and inode exhaustion?
Disk space measures storage capacity, while inodes represent the number of files the filesystem can store. A filesystem can report 100% inode usage even if significant disk space remains available.
Why does Oracle use /tmp?
Oracle Database, Oracle Grid Infrastructure, Oracle E-Business Suite, OPatch, AutoConfig, Java, and installation utilities use /tmp to store temporary working files during execution.
How can I prevent this issue?
Monitor filesystem and inode usage, remove obsolete temporary files regularly, enable approved automatic cleanup mechanisms, and investigate applications that generate excessive temporary data.
Related Articles
- Oracle Error Codes Guide
- ORA-01652: Unable to Extend Temp Segment
- ORA-01555: Snapshot Too Old
- About the Author
About the Author
Rana Abdul Wahid is an Oracle Database and Oracle E-Business Suite Consultant with more than 15 years of experience in Oracle Database Administration, Oracle E-Business Suite Application DBA, Oracle Cloud Infrastructure (OCI), Oracle RAC, Oracle Data Guard, RMAN Backup & Recovery, Linux/Unix Administration, MySQL, Microsoft SQL Server, PostgreSQL, and enterprise database management.
He regularly shares production-tested Oracle Database, Oracle E-Business Suite, Linux administration, and troubleshooting solutions based on real-world enterprise environments.
Conclusion
A 100% full /tmp directory is not simply a storage problem—it is often a symptom of application behavior, incomplete cleanup, inode exhaustion, or active processes holding temporary files open. Resolving the issue safely requires a structured approach that identifies the true cause before any cleanup is performed.
By verifying filesystem and inode usage, identifying large or orphaned files, checking for open deleted files, and understanding how Oracle software and Linux services use /tmp, administrators can restore normal operation while minimizing the risk of disrupting production workloads.
Never treat a full /tmp directory as a problem that can be solved by blindly deleting files. Investigate the root cause, confirm that temporary files are no longer in use, verify inode consumption, and monitor the filesystem after cleanup. A disciplined troubleshooting process reduces downtime and helps maintain a stable, secure, and reliable Linux environment for Oracle Database and Oracle E-Business Suite workloads.
Found this guide helpful? Visit the Oracle Error Codes Guide for more production-tested Oracle Database, Oracle E-Business Suite, Linux administration, and enterprise troubleshooting articles.
Comments
Post a Comment